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PREFACE 


This  document  supports  work  performed  by  the  Institute  for  Defense  Analyses  (IDA)  in  partial  fulfillment  of  the  task  titled 
“Analysis  of  System  Life  Cycle  Processes.”  The  work  was  sponsored  over  several  years  by  the  Office  of  the  Under  Secretary  of 
Defense  (Acquisition,  Technology,  and  Logistics),  Office  of  Interoperability  (10),  Office  of  Systems  Acquisition  (SA),  and  the  Office 
of  the  Director,  Test,  Systems  Engineering,  and  Evaluation. 

This  document  summarizes  the  myriad  of  life-cycle-process  standards;  capability  and  maturity  models;  process  improvement 
models;  and  appraisal,  assessment,  and  evaluation  methods  that  have  been  developed  and  used  by  industry  and  the  Department  of 
Defense.  The  Institute  for  Defense  Analyses  (IDA)  has  compiled  this  document  in  an  effort  to  clarify  and  document  the  background, 
purpose,  and  status  of  the  organizations,  standards,  models,  and  appraisal  methods  related  to  life  cycle  processes. 

This  document  was  reviewed  by  Mr.  Lance  Hancock  of  the  System  Evaluation  Division  of  IDA. 
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SUMMARY 
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A  multitude  of  international,  national,  and  industry-level  organizations  have  addressed  life  cycle  processes.  These  efforts  have 
been  formalized  in  many  different  life  cycle  process  standards,  models,  and  appraisal  methods.  The  accompanying  diagram  provides 
an  overview  of  how  these  standards,  models,  and  appraisal  methods  have  developed  over  the  past  decade,  as  well  as  the 
interrelationships  that  exist  between  them  (see  Appendix  A  for  acronym  and  abbreviation  definitions).  This  multitude  of  often 
overlapping  standards,  models,  and  assessment  methods  has  been  dubbed  “the  standards  quagmire.”1  This  document  presents 
information  on  the  various  international  and  national  organizations,  standards,  process  models,  capability  models,  maturity  models, 
process  improvement  models,  and  appraisal  methods  that  make  up  this  complex  picture  in  an  effort  to  provide  a  concise  snapshot  of 
the  backgrounds,  purpose,  status,  and  relationships  among  the  many  entities. 

The  document  is  organized  into  five  sections: 

•  Relevant  Organizations 

•  Standards  for  Life  Cycle  Processes 

•  Capability  and  Maturity  Models 

•  Process  Improvement  Models 

•  Appraisal  Methods 

Having  many  standards  and  models  leads  to  multiple  assessments,  evaluations,  and  appraisals  against  those  standards  and 
models.  DoD’s  Office  of  Systems  Engineering  sought  to  eliminate  many  duplicative  appraisals  and  improve  process  performance  for 
both  software  and  systems  development  by  sponsoring  a  Capability  Maturity  Model  Integration  (CMMI)  project  with  industry  and  the 
Software  Engineering  Institute.  This  document  provides  much  of  the  background  information  for  that  project. 


1  Sarah  Sheard,  “The  Frameworks  Quagmire,”  CrossTalk,  http://www.stsc.hill.af.mil/CrossTalk/1997/sep/frameworks.htm. 
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The  first  section  describes  the  relevant  organizations  involved  in  life  cycle  process  modeling. 
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Relevant  Organizations 


International  Organization  for  Standardization  (ISO) 
International  Electrotechnical  Commission  (IEC) 

ISO/IEC  Joint  Technical  Committee  1  (JTC1) 

Electronic  Industries  Alliance  (EIA) 

Institute  of  Electrical  and  Electronic  Engineers  (IEEE) 
International  Council  on  Systems  Engineering  (INCOSE) 
Software  Engineering  Institute  (SEI) 

Enterprise  Process  Improvement  Collaboration  (EPIC) 

Software  Process  Improvement  Capability  dEtermi nation 
(SPICE)  Project 

BOOTSTRAP  Institute 
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The  International  Organization  for  Standardization  is  an  organization  that  promotes  international  standardization  across  a 
broad  range  of  technical  and  functional  areas.  “The  mission  of  ISO  is  to  promote  the  development  of  standardization  and  related 
activities  in  the  world  with  a  view  to  facilitating  the  international  exchange  of  goods  and  services,  and  to  developing  cooperation  in 
the  spheres  of  intellectual,  scientific,  technological  and  economic  activity.”  ISO’s  focus  is  based  on  standardization’s  importance  in: 

•  Advancing  trade  liberalization 

•  Adapting  to  the  interdependence  of  sectors 

•  Fostering  global  communications  systems 

•  Defining  factors  of  new  technologies 

•3 

•  Improving  the  position  of  developing  countries 

Each  member  country  is  represented  in  ISO  by  its  national  standards  body.  ISO  has  been  involved  in  the  development  of 
standards  and  technical  reports,  the  most  notable  are  the  ISO  9000  family  of  standards.  This  family  of  standards  includes  ISO 
9000:2000,  Quality  management  systems — Fundamentals  and  vocabulary,  ISO  9001:2000,  Quality  management  systems — 
Requirements,  and  ISO  9004:2000,  Quality  management  systems — Guidelines  for  performance  improvements.  These  documents  are 
globally  accepted  by  organizations  for  conducting  international  trade.  ISO  also  develops  standards  in  conjunction  with  the 
International  Electrotechnical  Commission  (see  pages  10  and  1 1). 


2  “Introduction  to  ISO — What  is  ISO?,”  ISO  Web  site,  http://www.iso.ch/infoe/intro.html,  20  November  2000. 

3  “Introduction  to  ISO — Why  is  International  Standardization  Needed?,”  ISO  Web  site,  http://www.iso.ch/infoe/intro.html,  20  November  2000. 
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International  Organization  for 
Standardization  (ISO) 


Consists  of  standards  bodies  representing 
each  member  country 

Promotes  standardization  for  international 
commerce 

Involved  in  development  of  international 
standards  and  technical  reports,  for  example 
•  ISO  9000:2000 
ISO  9001:2000 
ISO  9004:2000 


A  simplified  version  of  ISO’s  organizational  structure4  is  illustrated  in  the  accompanying  diagram.  Within  this  overall 
organizational  structure,  the  technical  committees  (TCs)  are  the  bodies  that  perfonn  the  technical  work  necessary  for  the  development 
of  a  standard.  Through  a  specific  technical  committee  and  its  hierarchical  structure,  standards  are  developed  based  on  consensus, 
voluntary  involvement,  and  global-  and  industry-wide  solutions. 

ISO’s  175+  TCs  represent  specific  products  and/or  industries  and  perform  relevant  standardization  work.  Some  examples  of 
TCs  are  as  follows: 

•  TC  2,  Fasteners 

•  TC  20,  Aircraft  and  space  vehicles 

•  TC  176,  Quality  management  and  quality  assurance5 

Each  TC  performs  the  technical  work  for  a  specific  area  of  interest.  This  work  is  accomplished  within  a  hierarchical  structure 
on  both  the  international  and  national  levels.  At  the  international  level,  this  hierarchy  consists  of  subcommittees  and  working  groups, 
each  with  their  specific  focus.  At  the  national-level,  each  member  country  has  its  own  representative  national  body  that  is  responsive 
to  the  TC  of  interest,  a  national  technical  advisory  group  (TAG)  linked  to  a  specific  SC,  and  technical  groups  corresponding  to  each  of 
the  working  groups.  In  the  United  States,  the  national  body  is  the  American  National  Standards  Institute  (ANSI).  The  accompanying 
diagram  illustrates  the  ISO  TC’s  international-  and  national-level  components  and  their  interrelationships. 

IDA  is  an  active  member  of  the  U.S.  TAG  to  TC  176. 


4  Created  from:  “ISO’s  Structure,”  ISO  Web  site,  http://www.iso.cMso/en/aboutiso/isostmcture/isostr.html,  20  April  2001. 

5  “List  of  Technical  Committees,”  ISO  Web  site,  http://www.iso.ch/iso/en/stdsdevelopment/tc/tclist/TechnicalCommitteeList.TechnicalCommitteeList, 
26  November  2001. 
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ISO  Technical  Committee 
Structure 


175+  Technical  Committees 
perform  ISO’s  technical  work 
Each  possesses  its  own 

hierarchical  structure 

International  level 
Subcommittee  (SC) 

Working  group  (WG) 

National  level 

Technical  Advisory  Group  (TAG) 
Task  Group  (TG) 


_ 

1 

TG 

TG 
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The  International  Electrotechnical  Commission  consists  of  participating  countries,  each  of  which  “agrees  to  open  access  and 
balanced  representation  from  all  private  and  public  electrotechnical  interests  in  its  country,”  and  receives  the  right  to  full  participation 
in  the  preparation  and  publication  of  standards.6  The  IEC’s  mission  is  “to  promote,  through  its  members,  international  cooperation  on 
all  questions  of  electrotechnical  standardization  and  related  matters,  such  as  the  assessment  of  conformity  to  standards,  in  the  fields  of 
electricity,  electronics,  and  related  technologies.”7 8  In  order  to  support  this  mission,  the  IEC  strives  to: 

•  Meet  global  market  requirements  in  an  efficient  manner 

•  Maximize  use  of  its  standards  and  assessment  methodologies 

•  Improve  and  assess  products  and  services  on  the  basis  of  their  quality 

•  Set  parameters  for  the  interoperability  of  systems 

•  Improve  the  efficiency  of  industrial  processes 

•  Improve  human  health  and  safety 

8 

•  Protect  the  environment 

ISO  and  the  IEC  collaborate  on  Joint  Technical  Committee  1  (JTC1),  Information  Technology,  which  is  focused  on 
“standardization  in  the  field  of  information  technology.”  Joint  life  cycle  process  standards  include  ISO/IEC  12207:  Information 
Technology — Software  Life  Cycle  Processes,  ISO/IEC  15288:  Systems  Engineering — System  Life  Cycle  Processes,  and  ISO/IEC  TR 
15504:  Information  Technology — Software  Process  Assessment. 


6  “Inside  the  IEC,”  IEC  Web  site,  http://www.iec.ch/gnotel-e.htm,  30  November  2000. 

7  Ibid. 

8  Ibid. 
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International  Electrotechnical 
Commission  (IEC) 

Consists  of  member  countries 

Promotes  international  cooperation  in 
electrotechnical  standardization 

Participates  with  ISO  on  Joint  Technical 
Committee  1  (JTC1),  Information  Technology 

Involved  in  the  development  of  life  cycle 
process  standards  and  technical  reports,  for 
example 

•  ISO/IEC  12207 

•  ISO/IEC  15288 

•  ISO/IEC  TR  15504 


11 


The  accompanying  diagram  shows  the  hierarchical  structure  for  JTC1,  Information  Technology.  JTC1  is  divided  into  17  SCs 
according  to  the  specific  use  and  application  of  information  technology  within  different  sectors.  The  SC  of  greatest  relevance  to  this 
document  is  SC7 — Software  and  System  Engineering.  Presently,  the  Secretariat  of  SC7  is  held  by  the  Standards  Council  of  Canada 
and  its  membership  consists  of  28  participating  countries  and  18  observer  countries.9  Each  year,  SC7  conducts  one  international 
plenary  meeting  hosted  by  a  member  country. 

SC7  is  divided  into  1 1  working  groups  (WGs),  two  of  which  are  of  particular  importance  to  this  document:  WG7,  Life  Cycle 
Management,  and  WG10,  Process  Assessment.  The  convener  for  WG7  is  Doug  Thiele  of  Australia.  Alec  Dorling  of  the  United 
Kingdom  is  the  convener  for  WG  10.  Each  working  group  conducts  at  least  two  international  interim  meetings  each  year  in  member 
countries. 

The  national-level  involvement  represented  in  this  diagram  is  that  of  the  United  States.  The  Institute  of  Electrical  and 
Electronic  Engineers  (IEEE)  is  the  U.S.  member  organization  to  JTC1  SC7.  Operationally,  the  day-to-day  work  on  behalf  of  the 
United  States  is  performed  by  the  U.S.  TAG,  consisting  of  TGs  that  correspond  to  each  of  JTC1  SC7’s  WGs.  DoD  and  IDA 
participate  in  the  U.  S.  TAG  to  SC7,  Software  and  System  Engineering.  The  work  on  life  cycle  process  standards  and  assessments 
occurs  in  TG7,  Life  Cycle  Management,  and  TG10,  Process  Assessment.  U.S.  TAG  meetings  occur  two  to  three  times  a  year,  and  each 
TG  has  interim  meetings  as  necessary  to  follow  the  schedule  of  required  international  votes  on  work  products. 

IDA  is  an  active  member  of  SC7  and  participates  in  WG7  and  WG10  work. 


9  “JTC1  SC7 — Software  and  System  Engineering,”  ISO  Web  site,  http://www.iso.ch/en/std. .  .lPage.TechnicalCommitteeDetail?COMMID=40.html, 
26  November  2001. 
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The  Electronic  Industries  Alliance  represents  the  electronics  industry  through  a  federation  of  industry-related  sectors  and 
associations.10  The  EIA  fosters  connections  within  the  electronic  industries;  projects  power  and  influence  in  terms  of  mapping  the 
future  of  technology  and  public  policy;  provides  industry  and  market  research,  data,  analysis,  and  forecasts;  and  develops  standards 
important  to  the  electronic  industries.11 

EIA  consists  of  six  autonomous,  yet  united,  associations  relevant  to  the  electronic  industries.  These  associations  are  the 
Telecommunications  Industry  Association  (TIA),  the  Consumer  Electronics  Association  (CEA),  the  Electronic  Components, 
Assemblies  and  Materials  Association  (ECA),  the  Government  Electronics  and  Information  Technology  Association  (GEIA),  the 
JEDEC  Solid  State  Technology  Association,  and  the  Electronic  Industries  Foundation  (EIF).  The  GEIA’s  Systems  Engineering 
Committee  has  been  instrumental  in  developing  many  systems  engineering  standards. 

During  the  summer  of  1994,  the  EIA  established  an  EIA  Working  Group,  which  worked  toward  the  development  of  EIA 
Interim  Standard  (IS)  632:1994,  Processes  for  Engineering  a  System.  This  standard  was  the  commercial  equivalent  to 
MIL-STD-499B,  Systems  Engineering,  which  was  never  published  due  to  DoD’s  move  toward  the  use  of  commercial  standards.  The 
EIA  was  joined  in  this  effort  by  the  Aircraft  Industry  Association  (AIA),  DoD,  the  National  Security  Industries  Association  (NSIA), 
the  Institute  of  Electrical  and  Electronic  Engineers  (IEEE),  and  the  International  Council  on  Systems  Engineering  (INCOSE).  In  1998, 
EIA  632,  Pr ocesses  for  Engineering  a  System,  became  a  full  standard.  In  addition  to  632,  EIA  has  developed  other  standards  such  as 

interim  standard  EIA/IS  731,  Systems  Engineering  Capability  Model  (SECM)  and  Appraisal  Method.  The  EIA  also  coordinated  with 
the  IEEE  on  IEEE/EIA  12207:1996:  Standard  for  Information  Technology — Software  life  cycle  processes. 


10 


11 


“Benefits,”  EIA  Web  site,  http://www.eia.org/members/benefits/index.cfm,  30  November  2000. 
Ibid. 
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Electronic  Industries  Alliance 
(EIA) 


Federation  of  associations  and  sectors 
associated  with  the  electronics  industry 

Resource  and  voice  for  advancement  of  the 
electronics  industry 

Involved  in  development  of  systems 
engineering  and  software  engineering 
standards,  for  example 
•  EIA  632 
EIA/IS  731 
IEEE/EIA  12207 
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The  Institute  of  Electrical  and  Electronic  Engineers  draws  its  membership  from  professionals  and  students  who  are  working 
toward  or  have  already  established  a  certain  level  of  professional  competence  in  the  fields  of  electrical  engineering  or  information 
technology,  e.g.,  computer  engineering,  biomedical  technology,  telecommunications,  electric  power,  aerospace  and  consumer 
electronics.  The  IEEE  “helps  advance  global  prosperity  by  promoting  the  engineering  process  of  creating,  developing,  integrating, 
sharing,  and  applying  knowledge  about  electrical  information  technologies  and  sciences  for  the  benefit  of  humanity  and  the 

n 

profession.”  Interaction  among  its  members  and  IEEE  work  can  take  place  by  and/or  through  regional  bodies,  technical  societies, 
technical  councils,  society  chapters,  and  sections. 

The  IEEE  Standards  Association  is  an  international  membership  organization  that  develops  and  disseminates  standards.  The 
IEEE  was  an  original  member  of  the  EIA  Working  Group  founded  to  develop  EIA/IS  632:1994,  Processes  for  Engineering  a  System. 
The  IEEE  also  developed  IEEE  1220-1994,  Trial-Use  Standard  for  Applications  and  Management  of  Systems  Engineering  Process, 
which  later  became  IEEE  1220-1998,  IEEE  Standard  for  the  Application  and  Management  of  the  Systems  Engineering  Process.  IEEE 
coordinated  with  the  EIA  on  IEEE/EIA  12207: 1996,  Standard  for  Information  Technology — Software  life  cycle  processes. 


12  “Understanding  Membership,”  IEEE  Web  site,  http://www.ieee.org/membership/understanding.html,  26  November  2001. 

13  “About  the  IEEE,”  IEEE  Web  site,  http://www.ieee.org/about,  28  November  2000. 
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Institute  of  Electrical  and 
Electronic  Engineers  (IEEE) 

Professional  society 

Promotes  the  engineering  process  for 
electrical  and  information  technologies  and 
sciences 

Involved  in  development  of  systems 
engineering  and  software  engineering 
standards,  for  example 
•  IEEE  1220 
IEEE/EIA  12207 
EIA/IS  632 
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The  International  Council  on  Systems  Engineering  is  a  professional  organization  for  industry  and  government  professionals,  as 
well  as  academics,  with  knowledge  of  and  experience  in  systems  engineering.  “INCOSE  is  an  international  authoritative  body 
promoting  the  interdisciplinary  approach  and  means  to  enable  the  realization  of  successful  systems.”14  In  fostering  the  definition, 
understanding  and  practice  of  systems  engineering,  INCOSE  operates  with  the  following  goals: 

•  Serve  as  a  focal  point  for  the  distribution  of  knowledge 

•  Encourage  collaboration  in  education  and  research 

•  Set  standards  for  professional  integrity 

•  Augment  the  professional  status 

•  Promote  government  and  industry  support  for  research  and  education16 

In  1992,  INCOSE  sponsored  the  Capability  Assessment  Working  Group  (CAWG),  which  functions  within  INCOSE’s 
Measurement  Technical  Committee.  The  CAWG  was  chartered  to  develop  “a  method  for  assessing  and  improving  the  efficiency  and 
effectiveness  of  systems  engineering.”16  The  CAWG  developed  and  then  released  version  1.0  of  the  Systems  Engineering  Capability 
Assessment  Model  (SECAM)  and  the  SECAM  Assessment  Method  in  February  and  March  1994,  respectively.  INCOSE  was  also 
involved  with  the  development  of  EIA/IS  632,  Process  of  Engineering  a  System  (as  an  original  member  of  the  EIA  Working  Group) 
and  involved  with  EPIC  in  the  development  of  EIA/IS  731,  Systems  Engineering  Capability  Model  (SECM)  and  the  SECM  Appraisal 
Method. 


14  INCOSE  Web  site,  http://www.incose.org,  29  November  2000. 

15  “Welcome  to  INCOSE! —  Missions,  Goals,  and  Objectives,”  INCOSE  Web  site,  http://www.incose.org/intro.html,  26  November  2001. 

16  Systems  Engineering  Capability  Model  (SECM),  EIA/IS  731.1,  vol.  1,  p.  1. 
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International  Council  on 
Systems  Engineering  (INCOSE) 

Professional  organization 

Promotes  an  interdisciplinary  approach  to 
systems  engineering 

Chartered  the  CAWG  to  develop  an 
assessment  and  improvement  method  for 
systems  engineering 

>  Involved  in  development  of  systems 
engineering  standards,  capability  models,  and 
assessment  methods,  for  example 

•  EIA/IS  632 

•  SECAM 
EIA/IS  731 
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The  Software  Engineering  Institute  was  established  by  DoD  as  a  federally  funded  research  and  development  center 

(FFRDC)17  because  of  the  importance  of  quality  software,  delivered  on-time  and  within  budget,  to  the  development  and  procurement 

of  defense  systems.  In  order  to  facilitate  this  relationship  between  defense  systems  and  software,  SEI’s  mission  is  to  “provide 

1  8 

leadership  in  advancing  the  state  of  the  practice  of  software  engineering  to  improve  the  quality  of  systems  that  depend  on  software.” 
SEI  is  chartered  to: 

1 .  “Bring  the  ablest  professional  minds  and  the  most  effective  technology  to  bear  on  the  rapid  improvement  of  the  quality 
of  operational  software  in  systems  that  depend  on  software 

2.  Accelerate  the  reduction  to  practice  of  modem  software  engineering  techniques  and  methods 

3.  Promulgate  the  use  of  modern  techniques  and  methods  throughout  the  defense  community.”19 

During  the  early  1990s  the  SEI  produced  various  versions  of  the  Capability  Maturity  Model  for  Software  (the  “CMM”),  based 
on  “the”  vision  of  Watt’s  Humphrey,  the  first  director  of  the  SEI’s  Software  Process  Program.  An  assessment  method  was  also 
developed  for  the  CMM.  In  part  due  to  the  success  of  this  model,  the  SEI  provided  project  management  and  administrative  support  to 
EPIC  to  produce  the  Systems  Engineering-Capability  Maturity  Model  (SE-CMM)  and  the  SE-CMM  Appraisal  Method  (SAM),  which 
were  released  as  SEI  documents.  The  SEI  produced  other  capability  maturity  models  (CMMs)  and  assessment  methods,  such  as  the 
Integrated  Product  Development  (IPD)  CMM  and  the  People  CMM.  The  SEI,  with  government  and  industry  sponsorship,  ultimately 
developed  the  Capability  Maturity  Model  Integration  (CMMI)  and  its  appraisal  method. 


17  The  specific  contract  for  the  SEI  FFRDC  is  held  by  Carnegie  Mellon  University. 

18  “About  the  SEI —  Welcome,”  SEI/Carnegie  Mellon  Web  site,  http://www.sei.cmu.edu/about/about.html,  27  November  2001. 

19  “The  SEI  Charter,”  SEI/Carnegie  Mellon  Web  site,  http://www.sei.cmu.edu/about/overview/sei/charter.html,  27  November  2001. 
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Software  Engineering 
Institute  (SEI) 


FFRDC  associated  with  Carnegie  Mellon 
University 

Provided  program  management  and 
administrative  support  to  EPIC 

^Involved  in  development  of  capability  maturity 
models  (CMMs)  and  assessment  methods  in 
four  areas: 

Software 

Systems  Engineering 
Integrated  Product  Development 
People 
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The  Enterprise  Process  Improvement  Collaboration  evolved  from  INCOSE’s  CAWG,  with  project  management  and 
administrative  support  provided  by  the  Software  Engineering  Institute  of  Carnegie  Mellon  University.  EPIC’s  focus  was  on  the 
development  of  a  systems  engineering  capability  maturity  model.  EPIC  released,  as  SEI  documents,  the  Systems  Engineering- 
Capability  Maturity  Model  (SE-CMM)  and  the  SE-CMM  Appraisal  Method  Description  in  December  1994  and  March  1995, 
respectively. 
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Enterprise  Process  Improvement 
Collaboration  (EPIC) 


Collaboration  of  industry,  government,  and 
academia 

Focused  on  the  development  of  a  systems 
engineering  capability  maturity  model 

Involved  in  development  of  a  capability  maturity 
model  and  appraisal  method 
•  SE-CMM 

SE-CMM  Appraisal  Method  Description 
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The  Software  Process  Improvement  Capability  dEtermination  (SPICE)  project  was  formed  as  a  major  international 
collaborative  effort  to  develop  an  international  framework  standard  for  software  process  assessment.  The  project  was  carried  out  under 
the  auspices  of  ISO/IEC  JTC1  SC7  WGIO.20  It  began  unofficially  in  1990  and  was  made  official  in  June  of  1993. 21  It  no  longer  exists 
as  a  project,  although  web  sites  about  it  are  still  available. 

The  result  of  the  project  was  Technical  Report  ISO/IEC  TR  15504  (see  pages  98-99).  “SPICE”  has  been  used  to  refer  to 
ISO/IEC  15504,  although  it  is  not  appropriate  or  accurate  to  do  so. 

The  Software  Quality  Institute  of  Griffiths  University  in  Australia,  the  Software  Engineering  Institute  (SEI)  at  Carnegie 

Mellon  University  in  Pittsburgh,  and  the  European  Software  Institute  (ESI)  were  all  partners  in  the  SPICE  project  and  were  heavily 

22 

involved  in  the  field  trials  that  began  in  1995.  The  ESI  uses  the  assessment  model  in  their  Business  Improvement  Guides. 


20  Software  Quality  Institute,  Griffith  University,  Australia,  http://www-sqi.cit.gu 

21  SEI  SPICE  homepage,  http://www.sei.cmu.edu/iso-15504/ 

22  http://www.esi.es/Proiects/PI/overview.html,  12  February  2002. 


24 


Software  Process  Improvement 
Capability  dEtermination  (SPICE) 
PfQject 

International  effort  under  the  auspices  of 
ISO/IEC  JTC1  SC7  WG  10 
•  Started  in  1993 

Purpose  to  develop  a  framework  standard  for 
software  process  assessment 

Resulted  in  9-part  technical  report,  ISO/IEC  15504 
SPICE  no  longer  exists  as  a  formal  project 
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The  Bootstrap  Institute,  based  in  Brussels,  is  a  non-profit  organization  formed  by  some  of  the  participating  members  of  the 
completed  European  Software  Program  for  Research  in  Information  Technologies  (ESPRIT)  project. 

Although  the  Institute  offers  licensing,  assessor  accreditation,  and  training  and  maintains  the  BOOTSTRAP  data  base,  the 
Institute’s  main  objectives  are  the  development  and  promotion  of  the  Bootstrap  Methodology.  The  approach  of  the  Bootstrap 
Methodology  is  to  “determine  by  assessment  the  gap  between  the  current  process  state  and  the  desired  process  state  for  a  particular 
aspect  of  the  business  and  then  to  develop  an  improvement  plan  from  that  analysis.”  The  Bootstrap  Methodology  is  compliant  with 
ISO/IEC  15504,  ISO  9000,  and  the  CMM  capability  levels.  Its  current  release  is  Version  3. 2. 24 


23  From  the  European  Software  Institute  Dictionary,  http://www.esi.es/Help/Dictionarv/Defmitions/Bootstrap.html. 

24  The  Bootstrap  Institute  home  page,  http://www.  Bootstrap-institute.com/home.htm. 
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Bootstrap  Institute 


Non-profit  organization  that  promotes  the 
Bootstrap  methodology  for  software  process 
improvement 

Purpose  to  service  the  European  software 
industry  to  improve  its  competitiveness 

>  Offers 

Licensing 

Assessor  Accreditation 
•  Training  Curriculum 
The  BOOTSTRAP  Data  Base 
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The  next  section  describes  six  standards  for  life  cycle  processes. 
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Standards  for  Life  Cycle 
Processes 


Software  Life  Cycle  Processes  (ISO/IEC  12207) 

System  Life  Cycle  Processes  (ISO/IEC  15288) 

Quality  Management  Systems — Requirements, 
ISO  9001:2000 

Application  of  ISO  9001  to  Software,  ISO/IEC 
9000-3 

Processes  for  Engineering  a  System,  EIA  632 

Application  and  Management  of  Systems 
Engineering  Process,  IEEE  1220 
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ISO/IEC  12207:1995,  Standard  for  Information  Technology-Software  Life  Cycle  Processes,  is  the  product  of  ISO/IEC  JTC1, 
SC7,  WG7,  as  described  on  pages  12  and  13.  ISO/IEC  12207  establishes  a  common  framework  or  architecture  for  software  life  cycle 
processes  and  is  intended  to  provide  engineering  discipline  to  software  development  and  maintenance.  The  need  for  a  common 
framework  was  established  by  virtue  of  the  increasing  incorporation  of  software  into  systems  and  technologies,  as  well  as  the 
existence  of  multiple  standards,  procedures,  methods,  tools,  and  environments  for  use  in  software  development.  The  Guide  for 
ISO/IEC  12207  (Software  Life  Cycle  Processes)  was  published  as  Technical  Report  ISO/IEC  TR  15271  in  1998. 

Since  ISO/IEC  12207  was  initially  released  in  1995  and  expected  to  have  a  lifespan  of  25  to  30  years,  the  need  to  incorporate 
changes  into  it  was  anticipated  in  order  to  maintain  the  standard’s  relevance  to  industry  and  the  manufacture  of  software.  Accordingly, 
in  July  1999  WG7  released  its  “Vision  2020”  for  ISO/IEC  12207.  Central  to  this  endeavor  was  the  establishment  of  a  plan  for 
maintaining  this  standard  through  2020,  with  updates  of  the  standard  every  3  to  5  years.  The  first  Proposed  Draft  Amendment 
(PD AM)  was  released  in  1999  concurrent  with  the  concept  for  “Vision  2020.” 

WG7  has  agreed  to  a  second  amendment.  A  study  period  has  been  authorized  to  harmonize  ISO/IEC  12207  with  ISO/IEC 
15288,  which  is  currently  in  final  draft  international  standard  (FDIS)  (see  pages  36-39);  ISO/IEC  9000-3,  which  is  in  committee  draft 
(see  pages  4043);  and  ISO/IEC  15504,  which  is  in  various  stages  of  revision  (see  pages  98-99). 

IEEE  and  EIA  jointly  agreed  to  and  accepted  “clarifications,  additions,  and  changes”  to  ISO/IEC  12207:1995  and  documented 
these  in  their  own  joint  standard,  IEEE/EIA  12207.0-1996,  also  referred  to  as  US  12207,  which  is  intended  to  provide  industry  with  a 

25 

better  understanding  of  software  practices." 


25  IEEE/EIA  12207.0-1996,  Industry >  Implementation  of  International  Standard  ISO/IEC  12207:1995,  Standard  for  Information  Technology’ —  Software 
Life  Cycle  Processes,  March  1998. 
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Software  Life  Cycle  Processes 
ISO/I  EC  12207:1995 

Product  of  ISO/IEC  JTC1,  SC7,  WG7 

Purpose/Scope 

Need  for  this  standard 

>  Software  being  incorporated  into  many  systems  and  technologies 

>  Multiple  standards,  procedures,  methods,  tools,  and  environments 
used  for  software  development 

Establishes  a  common  framework  for  software  life  cycle 
processes 

Background/Status 

Initial  Release  —  August  1995 
•  1999  plan  for  amendment  and  revision  through  2020 
Harmonization  effort  underway 
Study  period  authorized  for  revision 
Second  amendment  agreed  to  by  WG7 
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ISO/IEC  12207:1995  consists  of  processes,  activities,  and  tasks  that  cover  the  life  cycle  of  software  from  conception  through 
retirement  and  demonstrates  the  relationships  among  these  processes.  The  processes  contained  in  ISO/IEC  12207  represent  a 
framework  that  covers  the  whole  software  life  cycle.  An  organization  may  seek  to  implement  all  of  the  standard’s  processes  or  only  a 
subset  of  those  processes  selected  via  the  standard’s  tailoring  process  to  meet  the  specific  software  needs  of  the  organization.  To 
successfully  perform  a  process,  all  of  the  activities  and  tasks  within  that  process  must  be  satisfied.  When  used  in  a  contract,  a 
minimum  set  of  processes,  activities,  and  tasks  needs  to  be  established  and  satisfied  in  order  for  a  supplier  to  be  in  compliance  with 
the  standard.  ISO/IEC  12207  describes  the  architecture  of  the  software  life  cycle  processes  but  does  not  specify  details  of  how  to 
implement  or  perform  the  activities  and  tasks  included  in  the  processes.  In  general,  the  “how  to”  implementation  is  left  to  lower-level 
standards. 

The  software  life  cycle  processes  are  captured  in  three  categories — Primary  Life  Cycle  Processes,  Supporting  Life  Cycle 
Processes,  and  Organization  Life  Cycle  Processes.  The  primary  processes  apply  to  primary  parties,  such  as  the  acquirer,  supplier, 
developer,  operator,  and  maintainer  involved  in  developing,  operating,  or  maintaining  software.  In  general,  the  software  life  cycle 
processes  are  initiated  by  the  Acquisition  process.  The  Supply  process  responds  and  initiates  the  development,  operations,  or 
maintenance  processes.  The  support  processes  are  those  that  contribute  to  the  success  and  quality  of  the  software  project  and  are 
“called”  by  other  processes.  Finally,  the  Organizational  processes  are  initiated  by  an  organization  to  establish,  implement,  or  improve 
other  software  life  cycle  processes. 
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ISO/IEC  12207:1995  List  of 
Processes 

Processes,  Activities,  and  Tasks  I 


Primary  Life  Cycle  Processes 

Acquisition  Process 
Supply  Process 
Development  Process 
Operation  Process 
Maintenance  Process 

Organizational  Life  Cycle 
Processes 

Management  Process 
Infrastructure  Process 
Improvement  Process 
Training  Process 


Supporting  Life  Cycle  Processes 

Documentation  Process 
Configuration  Management  Process 
Quality  Assurance  Process 
Verification  Process 
Validation  Process 
Audit  Process 

Problem  Resolution  Process 


**  Annex  A —  Tailoring  Process 
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After  experience  was  gained  in  the  use  of  ISO/IEIC  12207,  it  was  discovered  that  the  granularity  of  the  process  definitions  in 
ISO/IEC  12207:1995  made  it  difficult  to  derive  a  process  rating  component  for  the  purpose  of  process  assessment  and  improvement. 
The  Amendment  process  was  started  in  part  to  correct  this  and  to  provide  a  process  purpose  and  outcomes  so  that  ISO/IEC  12207  can 
become  a  Process  Reference  Model  in  accordance  with  the  new  requirements  of  ISO/IEC  15504  (see  pages  98-99).  Normative  Annex 
F  provides  the  process  reference  model  for  the  standard  and  gives  the  purpose  and  outcomes  for  each  process  in  the  standard.  The 
Amendment  is  also  in  accordance  with  the  architecture  of  the  existing  standard,  ISO/IEC  12207,  and  the  developing  standard, 
ISO/IEC  15288,  Systems  Engineering — System  Life  Cycle  Processes,  which  provides  a  process  purpose  and  outcomes  for  its  life  cycle 
processes. 
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ISO/IEC  12207=1995  Amendment 

Accommodates  requirements  of  present  and 
developing  SC7  standards  and  reports 

Changes  to  align  with  ISO/IEC  15504 
Resolves  granularity  of  the  process  definition 
Provides  process  purpose  and  outcomes 

Specific  text  changes  and  addition  of  3  annexes 

No  longer  prohibits  ISO/IEC  12207  use  for  off-the- 
shelf  software 

Adds  text  on  conformance  to  purposes  and  outcomes 

Annex  F:  Process  Reference  Model  from  ISO/IEC 
TR  1 5504  Part  2 

Clarification  that  lists  of  tasks  are  not  “exhaustive” 
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ISO/IEC  15288,  Systems  Engineering — System  Life  Cycle  Processes,  covers  the  life  cycle  of  a  man-made  system  from  concept 
through  retirement.  “It  provides  the  processes  for  acquiring  and  supplying  system  products  and  services  that  are  configured  from  one 
or  more  of  the  following  types  of  system  components:  hardware,  software,  and  humans.  In  addition,  the  framework  provides  for  the 
assessment  and  improvement  of  the  life  cycle.”  This  standard  is  designed  to  be  used  by  an  organization,  a  project  within  an 
organization,  or  an  acquirer  and  a  supplier  via  an  agreement 

As  of  this  publication,  ISO/IEC  15288  is  in  Final  Draft  International  Standard  (FDIS)  form  and  being  balloted.  When  voted  on 
and  passed  by  two  thirds  of  the  member  bodies,  it  becomes  an  International  Standard.  This  is  expected  by  fall  2002. 

An  effort  is  underway  to  hannonize  many  of  the  standards  discussed  in  this  document  once  ISO/IEC  15288  is  published. 
Resolution  629  at  the  Nagoya  SC7  Plenary  meeting  states: 

JTC1/SC7  intends  to  initiate  a  revision  of  ISO/IEC  15288  and  ISO/IEC  12207  for  harmonization  between  these 
standards  and  also  ISO  9000-3  and  ISO/IEC  TR  15504  as  soon  as  ISO/IEC  15288  and  ISO/IEC  12207  PD  AM  are 
published.  To  prepare  for  this  work  JTC1/SC7  instructs  its  WG7  to  initiate  a  study  period  once  a  successful  FCD  and 
FDAM  ballot  have  been  completed. 

The  SC7  also  created  an  Ad  Hoc  System  Engineering  Study  Group  that  surveyed  the  utility  of  the  systems  engineering 
standards  for  an  organization  to  implement  a  systems  engineering  approach.  It  determined  that  an  organization  would  need  ISO/IEC 
15288,  EIA  632  (see  pages  48-51),  and  IEEE  1220  (see  pages  52-55).  ISO/IEC  15288  defines  the  processes  needed  during  a  system’s 
life  cycle,  EIA  632  defines  the  set  of  requirements  for  engineering  a  system,  and  IEEE  1220  defines  the  systems  engineering  process 
itself.  An  approach  has  been  proposed  for  harmonizing  these  three  documents  and  making  them  consistent  with  each  other. 


26  ISO/IEC  15288  FCD,  Introduction. 
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Software  Life  Cycle  Processes 
ISO/I  EC  12207:1995 

Product  of  ISO/IEC  JTC1,  SC7,  WG7 

Purpose/Scope 

Need  for  this  standard 

>  Software  being  incorporated  into  many  systems  and  technologies 

>  Multiple  standards,  procedures,  methods,  tools,  and  environments 
used  for  software  development 

“Establishes  a  common  framework  for  software  life  cycle 
processes” 

Background/Status 

Initial  Release — August  1995 
•  1999  plan  for  amendment  and  revision  through  2020 
Harmonization  effort  underway 
Study  period  authorized  for  revision 
Second  amendment  agreed  to  by  WG7 
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The  processes  in  ISO/IEC  15288  are  grouped  into  four  categories:  Agreement,  Enterprise,  Project,  and  Technical. 

For  each  process  listed,  a  process  purpose  and  outcomes  are  given  in  order  to  establish  a  Process  Reference  Model  in 
accordance  with  requirements  of  ISO/IEC  15504-2,  Information  technology  — Software  Process  Assessment — Part  2:  A  Reference 
Model  for  Processes  and  Process  Capability.  Activities  are  also  given  for  each  process,  but  it  is  the  accomplishment  of  the  outcomes 
that  gives  evidence  that  the  requirements  of  an  organization’s  declared  set  of  processes  (those  that  apply  to  its  business  objectives)  are 
being  met. 

A  Guide  is  also  being  developed  for  this  standard  that  gives  greater  detail  about  implementation  of  these  processes.  ISO/IEC 
TR  19760,  Guide  for  ISO/IEC  15288  (System  Life  Cycle  Processes)  is  currently  in  ballot  for  combined  WD/PDTR  Registration,  due 
5  June  2002.  The  intention  is  for  it  to  be  published  shortly  after  the  International  Standard. 


38 


ISO/IEC  15288  FDIS  List  of 
Processes 

Process  Outcomes  and  Activities 


Enterprise  Processes 

Enterprise  Environment 
Management 
Investment  Management 
System  Life  Cycle  Processes 
Management 
Resource  Management 
Quality  Management 

Project  Processes 

Project  Planning 
Project  Assessment 
Project  Control 
Decision  Making 
Risk  Management 
Configuration  Management 
Information  Management 


Agreement  Processes 

Acquisition 

Supply 

Technical  Processes 

Stakeholder  Requirements  Definition 

Requirements  Analysis 

Architectural  Design 

Implementation 

Integration 

Verification 

Transition 

Validation 

Operation 

Maintenance 

Disposal 
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ISO  9001:2000,  Quality  Management  Systems — Requirements,  is  perhaps  the  best  known  international  standard  on  the  market. 
This  revision  of  the  1994  version  took  a  process  focus,  and  so,  is  pertinent  to  this  document.  ISO  9001:2000  is  also  ISO’s  “umbrella” 
publication  to  which  all  other  ISO  standards  need  to  conform  in  some  manner.  This  is  particularly  true  for  vocabulary,  which  is  found 
in  its  accompanying  publication,  ISO  9000:2000,  Quality  Management  Systems — Fundamentals  and  Vocabulary.  These  ISO 
vocabulary  directives  proved  difficult  to  follow  in  some  instances  in  the  development  of  ISO/IEC  15288.  For  example,  “system”  in  the 
sense  of  a  quality  management  system  has  a  different  connotation  than  “system”  in  the  sense  of  an  aerospace  system.  Definitions  for 
“verification”  and  validation”  were  also  established  in  ISO  9000:2000  and  needed  to  be  followed  in  the  development  of  ISO/IEC 
15288.  Of  necessity,  some  deviations  in  definitions  were  used  and  these  appear  in  the  FDIS  of  ISO/IEC  15288.  How  these  vocabulary 
issues  are  resolved  at  the  ISO  level  remains  to  be  seen. 


40 


Quality  Management  Systems — 
Requirements,  ISO  9001:2000 

Product  of  ISO  Technical  Committee  176 

•  Subcommittee  2,  Quality  Systems 

>  Purpose 

“Promotes  the  adoption  of  a  process  approach  when 
developing,  implementing,  and  improving  the 
effectiveness  of  a  quality  management  system,  to 
enhance  customer  satisfaction  by  meeting  customer 
requirements” 

>  Background/Status 

Third  Edition  of  International  Standard  published 
13  December  2000 


41 


The  list  of  processes  in  this  standard  differs  from  those  in  the  other  standards  and  models  described  in  this  document.  These 
processes  are  associated  with  a  quality  management  system,  whereas  the  other  processes  were  associated  with  a  part  of  the  life  cycle 
of  a  system.  Still,  because  of  the  globally  pervasive  nature  of  ISO  9001:2000,  most  standards  and  models  that  organizations  choose  to 
use  in  addition  to  900 1  will  need  to  map  to  these  processes  in  some  fashion. 
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ISO  9001:2000  List  of  Processes 


Management  Responsibility 

Resource  Management 

Measurement,  Analysis,  and  Improvement 

Product  Realization 

Planning  of  Product  Realization 
Customer-Related  Processes 
Design  and  Development 
•  Purchasing 

Production  and  Service  provision 

Control  of  Monitoring  and  Measuring  Devices 
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ISO  TC176  coordinated  with  JTC1  SC7  on  the  fate  of  ISO  9000-3:1997,  Quality  Management  Assurance  Standards — Part  3: 
Guidelines  for  the  Application  of  IS09001:1994  to  Development,  Supply,  Installation,  and  Maintenance  of  Computer  Software. 
TC176  eventually  agreed  to  transfer  ISO  9000-3:1997  to  SC7  for  the  creation  of  the  new  version  to  accompany  ISO  9001:2000.  This 
new  document  is  currently  in  Final  Committee  Draft  (FCD)  form  and  out  for  ballot.  It  is  called  ISO/IEC  9000-3,  Software  and  System 
Engineering — Guidelines  for  the  Application  of  ISO  900 1 :2000  to  Software.  It  is  currently  scheduled  for  publication  in  early  2003 . 
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Application  of  ISO  9001:2000 
to  Software,  ISO/I  EC  9000-3 

> Product  of  ISO/IEC  JTC1  SC7  WG7 

>  Purpose 

“This  International  Standard  provides  guidance  in 
applying  the  requirements  of  ISO  9001:2000  where 
computer  software  design,  development,  supply, 
installation  and  maintenance  are  elements  of  the 
business  of  an  organization” 

>  Backg  rou  nd/Status 

Ownership  transferred  from  ISO  TCI  76  to  JTC1  SC7 

Final  Committee  Draft  (FCD)  out  for  ballot  due  May 
2002 

Expected  publication  1st  quarter  2003 
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Since  SC7  is  now  involved  in  redoing  ISO  9000-3,  the  product  realization  processes  are  aligned  with  the  processes  in 
ISO/IEC  12207. 
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ISO/IEC  9001-3  List  of  Processes 

Management  Responsibility 
Resource  Management 
Product  Realization 

Planning  of  Product  Realization 
Customer-Related  Processes 
Design  and  Development 
•  Control  of  Production  and  Service  Provision 
Validation  of  Processes 
Purchasing 
Customer  property 
Identification  and  Traceability 
Preservation  of  Product 
Control  of  Monitoring  and  Measuring  Devices 

Measurement,  Analysis  and  Improvement 
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The  original  EIA  Interim  Standard  (IS)  632,  Systems  Engineering,  was  published  by  the  EIA  in  December  1994 — the  result  of 
a  working  group  made  up  of  members  from  EIA,  IEEE,  INCOSE,  AIA,  NSIA,  and  DoD.  This  interim  standard  was  the 
commercialized  version  of  the  MIL-STD-499B,  Systems  Engineering.  MIL-STD-499B  was  only  published  in  draft  form  as  it  was 
overcome  by  the  events  of  DoD’s  acquisition  reform  move  to  commercial  standards.  MIL-STD-499B  had  been  drafted  by  the  Air 
Force  System’s  Command  Directorate  of  Systems  Engineering  and  the  Defense  Systems  Management  College  (DSMC)  Systems 
Engineering  Department. 

Work  on  the  ANSI/EIA  632  standard,  Processes  for  Engineering  a  System ,  began  in  1997  with  the  intent  for  it  to  be  the  early 
implementation  of  systems  engineering  that  would  later  be  covered  by  ISO/IEC  15288.  The  processes  in  EIA  632  describe  “what  to 
do”  with  respect  to  the  processes  for  engineering  a  system,  which  is  the  next  level  down  from  the  ISO/IEC  15288  level  of  system  life 
cycle  processes. 

INCOSE  participated  in  the  creation  of  this  standard  with  the  G47  Systems  Engineering  committee  of  the  Government 
Electronic  and  Infonnation  Technology  Association  (GEIA)  of  the  EIA. 
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Processes  for  Engineering  a 
System,  ANSI/EIA  632 

Product  of  ANSI  and  ElAwith  INCOSE 
>  Purpose/Scope 

“Provide  an  integrated  set  of  fundamental  processes 
to  aid  a  developer  in  the  engineering  and 
reengineering  of  a  system” 

>Background/Status: 

Initial  release — EIA/IS  632,  December  1994 

EIA  Working  Group  (EIA,  IEEE,  AIA,  INCOSE,  NSIA,  DoD) 
Commercialized  version  of  MIL-STD-499B 

ANSI/EIA  release  in  1998 

Intended  to  be  early  U.S.  implementation  of  systems 
engineering  to  be  covered  by  ISO  15288 
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The  processes  for  engineering  a  system  are  divided  into  five  categories:  Acquisition  and  Supply,  Technical  Management, 
System  Design,  Product  Realization,  and  Technical  Evaluation.  Acquisition  and  Supply  processes  are  used  by  the  developer  to  arrive 
at  an  agreement  with  another  to  accomplish  specific  work  and  deliver  required  products.  The  Technical  Management  processes  area  is 
used  to  plan,  assess,  and  control  the  technical  work  required  to  satisfy  the  established  agreement.  The  System  Design  processes  are 
used  to  convert  agreed-upon  requirements  of  the  acquirer  into  a  set  of  realizable  products  that  satisfy  acquirer  and  other  stakeholder 
requirements.  Product  Realization  processes  are  used  to  convert  the  specified  requirements  and  other  design  solution  characterizations 
into  either  a  verified  end  product  or  a  set  of  end  products  in  accordance  with  the  agreement  and  other  stakeholder  requirements.  And, 
finally,  the  Technical  Evaluation  processes  are  invoked  by  one  of  the  other  processes  for  engineering  a  system.  These  consist  of 
systems  analysis,  requirements  validation,  system  verification,  and  end  product  validation  processes. 

Each  process  is  described  by  requirements.  The  influence  of  the  enterprise  and  the  project  are  discussed  on  the  application  of 
the  processes,  as  well  as  the  influence  of  other  external  factors.  The  engineering  life  cycle  is  also  discussed  in  the  context  of  the 
project  life  cycle. 
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ANSI/EIA  632  List  of  Processes 

Processes  and  Requirements  j 


Fundamental  Processes  for  Engineering  a  System 


Acquisition  and  Supply 

-  Supply 

-  Acquisition 
Technical  Management 

-  Planning 

-  Assessment 

-  Control 
System  Design 

-Requirements  Definition 
-Solution  Definition 


Product  Realization 

-  Implementation 

-  Transition  to  Use 
Technical  Evaluation 

-  Systems  Analysis 

-  Requirements  Validation 

-  System  Verification 

-  End  Products  Validation 
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IEEE  1220,  Trial-Use  Standard  for  Application  and  Management  of  the  Systems  Engineering  Process,  was  published  on 
28  February  1995.  The  authorized  standard  was  published  in  1998,  its  major  differences  being  greater  emphasis  on  software  and  on 
engineering  the  system  for  humans.  The  latter  standard  simplified  the  systems  breakdown  structure,  clarified  confonnance  statements, 
and  broke  functional  analysis  into  context  analysis  and  functional  decomposition  for  clarity  as  well. 

IEEE  1220  gives  the  next  level  of  detail  below  the  process  requirements  described  in  EIA  632.  The  processes  are  described 
more  at  the  task  or  application  level.  IEEE  1220  also  does  not  worry  about  “who  does  what”  as  some  of  the  other  standards  do  with 
the  “acquirer-supplier”  concepts. 
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Application  and  Management  of  the 
SE  Process,  IEEE  1220:  1998 


Product  of  IEEE 
>  Purpose/Scope 

“Provide  standard  for  managing  a  system  from  initial 
concept  though  development,  and  define  life  cycle 
disposal” 

>Background/Status 

Initial  release — February  1995  (“Trial-Use  Standard”) 

Intent  was  to  merge  with  EIA/IS  632  to  form  one  ANSI 
standard,  published  jointly  by  EIA,  IEEE,  and  INCOSE 

Most  recent  release — December  1998  (full,  authorized 
standard) 
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The  processes  in  IEEE  1220,  Standard  for  Application  and  Management  of  the  Systems  Engineering  Process,  are  divided  into 
those  for  the  life  cycle  and  those  of  the  Systems  Engineering  Process  (SEP).  The  process  descriptions  provide  “interdisciplinary  tasks 
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that  are  required  throughout  a  system’s  life  cycle  to  transform  customer  needs,  requirements,  and  constraints  into  a  system  solution.” 


27 


IEEE  1220,  Standard  for  Application  and  Management  of  the  Systems  Engineering  Process,  IEEE  Computer  Engineering  Society,  Software  Engineering 
Standards  Committee,  1998. 


54 


IEEE  1220:  1998  List  of  Processes 

Systems  Engineering  Process  (SEP) 

Requirements  Analysis 

Requirements  Validation 

Functional  Analysis 

- "Each  life  cycle  process  ... 

ms  itself  a  system  in  that  products  must 
"oe  developed  to  fulfill  the  purpose-^^* 

Functional  Verification 

of  the  life  cycle  process. "r — 

Synthesis  ^ 

Design  Verification 

Systems  Analysis 

Life  Cycle  Processes 

Control  , 

Development 

M^'/V’ply  SEP  throughout'^^' 

Manufacturing 

Test 

system  life  cvcle 

Distribution 

Operation 

Support 

Training 

Disposal 
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The  following  section  of  this  document  describes  capability  and  maturity  models  for  life  cycle  processes. 
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Capability  and  Maturity  Models 

Capability  Maturity  Model  for  Software  (SW-CMM) 

Systems  Engineering  Capability  Maturity  Model 
(SE-CMM) 

Systems  Engineering  Capability  Model  (SECM) 
(EIA/IS  731.1) 

Integrated  Product  Development  Capability 
Maturity  Model  (IPD-CMM) 

Capability  Maturity  Model  Integration  (CMMI) 

integrated  Capability  Maturity  Model  (iCMM) 
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The  concept  of  maturity  levels  originated  in  the  quality  and  continuous  improvement  community  with  Philip  Crosby. 
Beginning  in  the  1970s,  Crosby  was  one  of  the  “Quality  Gurus,”  along  with  J.  Edwards  Deming  and  J.  M.  Juran.  Crosby  tended  to 
focus  on  the  management  responsibility  for  quality.  In  his  book,  Quality  is  Free,  Crosby  describes  the  improvement  process  as 
taking  place  over  live  stages:  Uncertainty,  Awakening,  Enlightenment,  Wisdom,  and  Certainty.  The  models  discussed  in  this  section 
use  this  concept,  although  the  maturity  or  capability  levels  are  more  specific  than  these  stages. 


28  Philip  B.  Crosby,  Quality  is  Free:  The  Art  of  Making  Quality  Certain,  McGraw-Hill  Book  Company,  1979. 


58 


Crosby’s  Quality  Management 
Maturity  Stages 


1.  Uncertainty 

2.  Awakening 

3.  Enlightenment 

4.  Wisdom 

5.  Certainty 
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During  the  early  1990s,  the  SEI  produced  various  versions  of  the  Capability  Maturity  Model®  (CMM®)29  for  Software,  based 
on  the  vision  of  Watts  Humphrey,  the  first  director  of  the  SEI’s  Software  Process  Program.  This  model  uses  a  staged  representation 
that  has  five  maturity  levels  that  an  organization  can  achieve  in  its  software  process  improvement  efforts.  An  assessment  framework, 
model,  and  method  were  all  developed  for  the  CMM.  The  appraisal  methods  are  described  in  the  next  section  of  this  document.  The 
Software  CMM  gained  wide  acceptance  and  the  SEI  has  collected  much  information  and  data  on  its  use  and  results  of  its  use.  Authors 
include  Mark  C.  Paulk,  Bill  Curtis,  Mary  Beth  Chrissis,  and  Charles  V.  Weber. 

When  people  speak  of  “the  CMM”  or  the  “SEI  CMM,”  it  is  this  model  to  which  they  refer.  The  current  policy  in  DoD  5000.2 

states: 

Select  contractors  with  domain  experience  in  developing  comparable  software  systems;  with  successful  past 
performance;  and  with  a  mature  software  development  capability  and  process.  Contractors  performing  software 
development  or  upgrade(s)  for  use  in  an  ACAT  I  or  ACAT  IA  program  shall  undergo  an  evaluation,  using  either  the 
tools  developed  by  the  Software  Engineering  Institute  (SEI),  or  those  approved  by  both  the  DoD  Components  and  the 
Deputy  Under  Secretary  of  Defense  (Science  and  Technology)  (DUSD(S&T)).  At  a  minimum,  full  compliance  with 
SEI  Capability  Maturity  Model  Level  3,  or  its  equivalent  in  an  approved  evaluation  tool,  is  the  Department's  goal.30 

The  Software  CMM,  Version  2.0  Draft  C,  was  used  as  a  source  model  for  the  Capability  Maturity  Model  Integration  (CMMI) 
(see  pages  76-79).  The  SEI  will  no  longer  publish  any  updates  to  the  SW-CMM  model  or  training  materials;  however,  the  SEI  will 
offer  its  course,  Introduction  to  SW-CMM ,  for  2  years  after  the  publication  of  CMMI  V 1 . 1 .  The  SEI  will  not  withdraw  the  rights  of  the 
organizations  that  are  transition  partners  to  deliver  the  Introduction  to  SW-CMM  training  beyond  the  2-year  period  to  select 

3 1 

organizations  requesting  on-site  delivery.  New  instructors  may  be  authorized,  if  needed. 


29  Capability  Maturity  Model®,  CMM®,  Capability  Maturity  Model  Integration™,  and  CMMI™  are  registered  terms  belonging  to  the  SEI  and  Carnegie  Mellon 
University.  For  simplicity,  they  are  not  always  marked  as  such  in  this  document. 

30 

DoD  5000.2,  Mandatory  Procedures  for  Major  Defense  Acquisition  Programs  (MDAPs)  And  Major  Automated  Information  System  (MAIS)  Acquisition 
Programs,  June  2001. 

31  How  Will  Sunsetting  of  the  Software  CMM  be  Conducted?,  SEI  Web  site,  http://www.sei.cmu.edu/cmmi/comm/sunset.html. 
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Capability  Maturity  Model  for 
Software  (SW-CMM) 

Product  of  SEI 
>  Purpose/Scope 

Used  by  organizations  for  appraising  the  maturity  of 
their  software  processes  and  for  identifying  practices 
that  will  increase  the  maturity  of  those  processes 

Background/Status 

CMM  vl .0  released  in  August  1991 ,  vl.1  in  February 
1993 

Version  2.0  Draft  C  as  source  document  for  CMMI 

Staged  Representation 
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The  five  maturity  levels  of  the  CMM  for  Software  are  described  behaviorally  as  follows: 

1 .  Initial,  where  processes  are  performed  in  an  ad  hoc  manner 

2.  Repeatable,  where  discipline  is  introduced  into  the  processes  with  policies  and  procedures 

3.  Defined,  where  standard  processes  are  documented 

4.  Managed,  where  quantitative  quality  goals  are  set  for  products  and  processes 

5.  Optimizing,  where  the  entire  organization  is  focused  on  continuous  improvement 

Each  Maturity  Level  has  several  Key  Process  Areas  associated  with  it  “that  indicate  the  areas  an  organization  should  focus  on 
to  improve  its  software  process.  Key  process  areas  identify  the  issues  that  must  be  addressed  to  achieve  a  maturity  level.”  Each  Key 
Process  Area  has  a  set  of  related  activities  “that,  when  performed  collectively,  achieve  a  set  of  goals  considered  important  for 
enhancing  process  capability.”  These  activities  are  described  in  terms  of  goals  and  key  practices. 


32  Paulk,  Mark  C.,  Bill  Curtis,  Mary  Beth  Chrissis,  and  Charles  V.  Weber,  Capability  Maturity  Model  for  Software,  Version  1.1,  Technical  Report,  CMU/SEI- 
93-TR-024,  February  1993. 

33  Ibid. 

34  Paulk,  Mark  C.,  Charles  V.  Weber  ,  Suzanne  Garcia,  Mary  Beth  Chrissis,  and  Marilyn  Bush,  Key  Practices  of  the  Capability  Maturity  Model,  Version  1.1, 
Technical  Report,  CMU/SEI-93-TR-025,  February  1993. 
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SW-CMM  List  of  Processes 

Goals  and  Key  Practices  1 


Requirements  Management 
Software  Project  Planning 
Software  Project  Tracking  and 
Oversight 

Software  Subcontract 
Management 

Software  Quality  Assurance 
Software  Configuration 
Management 


Organization  Process  Focus 
Organization  Process  Definition 
Training  Program 
Integrated  Software  Management 
Software  Product  Engineering 
Intergroup  Coordination 
Peer  Reviews 

Qualitative  Process  Management 
Software  Quality  Management 
Defect  Prevention 
Technology  Change  Management 
Process  Change  Management 
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The  Systems  Engineering  Capability  Maturity  Model  (SE-CMM)  is  the  product  of  the  SE-CMM  Project,  which  comprises 
individuals  from  industry,  academia,  and  government  under  the  auspices  of  the  SEI.  The  industry  collaboration  on  the  project  became 
Enterprise  Process  Improvement  Collaboration  (EPIC)  (see  pages  22-23  of  this  document).  The  SE-CMM  “describes  the  essential 
elements  of  an  organization’s  systems  engineering  process  that  must  exist  to  ensure  good  systems  engineering.  In  addition,  the 
SE-CMM  provides  a  reference  for  comparing  actual  SE  practices  against  these  essential  elements.”  Like  other  CMMs,  it  does  not 
specify  a  particular  process  model  or  sequence  of  processes.  The  model  encompasses  all  phases  of  the  system  life  cycle  and  was 
designed  to  help  organizations  “improve  their  practice  of  SE  through  self-assessment  and  guidance  in  the  use  of  statistical  process 
control  principles.”  Use  of  the  model  for  supplier  selection  was  discouraged. 

When  the  SECM  was  later  released,  EPIC  no  longer  supported  the  SE-CMM,  which  was  merged  with  the  SECAM  to  form  the 
SECM,  EIA/IS  731  (see  pages  68-71).  Current  effort  in  this  area  is  now  part  of  the  CMMI  project.36 


35  A  Systems  Engineering  Capability  Maturity  Model"',  Version  1.1,  CMU/SEI-95-MM-003,  November  1995. 

36  http://www.sei.cmu.edu/cmm/se-cmm.html 
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Systems  Engineering  Capability 
Maturity  Model  (SE-CMM) 

Product  of  EPIC 
Purpose/Scope 

Describes  the  essential  elements  of  an  organization’s 
systems  engineering  process  that  must  exist  to  ensure 
good  systems  engineering  and  encompasses  all 
phases  of  the  product  life  cycle 

Background/Status 

Version  1.0  of  SE-CMM  released  December  1994 
SE-CMM  v.1.1  released  November  1995 

Merged  with  the  SECAM  to  create  the  SECM  and  no 
longer  supported 
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The  SE-CMM  is  organized  around  the  process  capability  aspect,  which  included  the  generic  practices,  and  the  domain  aspect, 
which  included  the  base  practices.  The  base  practices  are  organized  into  process  areas,  which  in  turn  are  grouped  into  Engineering, 
Organizational,  and  Project  categories.  Each  process  area  was  ranked  at  a  certain  maturity  level,  based  on  implementation  of  the 

'XI 

generic  practices.  The  generic  practices  constituted  five  capability  levels: 

Level  0  Not  Performed 

Level  1  Performed  Informally 

Level  2  Planned  and  Tracked 

Level  3  Well-Defined 

Level  4  Quantitatively  Controlled 

Level  5  Continuously  Improving 

The  use  of  these  generic  practices  made  this  model  function  like  the  continuous  representation  of  later  models. 


37  A  Systems  Engineering  Capability  Maturity  Model,  Version  1.1,  CMU/SEI-95-MM-003,  November  1995. 
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SE-CMM  List  of  Processes 


Process  Areas  and  Base  Practices 


Engineering  Process  Areas 

Analyze  Candidate  Solutions 
Derive  and  Allocate 
Requirements 
Evolve  System  Architecture 
Integrate  Disciplines 
Integrate  System 
Understand  Customer  Needs 
Verify  and  Validate  System 

Project  Process  Areas 
Ensure  Quality 
Manage  Configurations 
Manage  Risk 

Monitor  and  Control  Technical  Effort 
Plan  Technical  Effort 


Organizational  Process  Areas 

Define  Organization's  Systems 
Engineering  Process 
Improve  Organization's  Systems 
Engineering  Process 
Manage  Product  Line  Evolution 
Manage  Systems  Engineering  Support 
Environment 

Provide  Ongoing  Skills  and  Knowledge 
Coordinate  with  Suppliers 
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EPIC’s  SE-CMM  and  INCOSE’s  SECAM  (see  pages  102-103)  were  combined  into  a  single  model  and  assessment  method 
under  the  auspices  of  a  project  charted  under  the  EIA  G-47  Systems  Engineering  Committee.  The  result  of  the  project  was  that  EIA 
published  the  EIA  Interim  Standard  (EIA/IS)  731-1,  Systems  Engineering  Capability  Model  (SECM)  and  EIA/IS  731-2,  Systems 
Engineering  Capability  Model  (SECM)  Appraisal  Method.  Thus,  Volume  1  contains  the  model  and  Volume  2  is  the  appraisal  method. 
The  model  contains  Generic  Practices  and  Generic  Attributes  that  are  grouped  into  the  four  levels  of  capability  above  level  1,  which 

•  38 

contains  none. 

1 .  Perfonned 

2.  Managed,  where  activities  are  planned  and  tracked 

3.  Defined,  where  activities  are  performed  according  a  well-defined  process  using  approved  versions  of  standard  and 
documented  processes  (may  be  tailored) 

4.  Measured,  where  measurement  is  applied  to  the  processes 

5.  Optimizing,  where  quantitative  performance  goals  for  process  effectiveness  and  efficiency  based  on  business  goals  are 
established 

The  SECM  was  originally  published  as  an  interim  standard  by  the  EIA  because  it  was  to  be  used  as  a  source  model  for  the 
CMMI  (see  pages  76-79)  and  would  be  canceled  once  the  CMMI  was  published.  Changes  in  the  rules  and  regulations  governing  EIA 
standards  has  eliminated  the  “Interim  Standard”  designation,  so  it  will  be  published  as  a  full  standard. 


38  EIA/IS  731.1,  Systems  Engineering  Capability  Model  (SECM). 
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Systems  Engineering  Capability 
Model  (SECM)  (EIA/IS  731) 

Product  of  EIA,  INCOSE,  and  EPIC 
Purpose/Scope 

“To  support  the  development  and  improvement  of 
systems  engineering  capability” 

Background/Status 

Reflects  March  1996  initiative  to  merge  SECAM  and 
SE-CMM 

Intended  to  complement  the  use  of  EIA  632  and 
IEEE  1220 

EIA/IS  731 ,  Draft  Version  1 .0 
Continuous  Representation 
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The  processes  in  EIA/IS  731  (SECM)  are  listed  within  three  systems  engineering  focus  areas:  Technical,  Management,  and 
Environment.  Each  process  has  specific  practices  and  generic  characteristics  associated  with  it. 
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EIA/IS  731  (SECM)  List  of 


Focus  Areas,  Specific 
Practices,  and  Generic 
Characteristics 


Systems  Engineering 
Technical  Focus  Areas 

Define  Stakeholder  and 
System  Level  Requirements 
Define  Technical  Problem 
Define  Solution 
Assess  and  Select 
Integrate  System 
Verify  System 
Validate  System 


Systems  Engineering 
Management  Focus  Areas 

Plan  and  Organize 
Monitor  and  Control 
Integrate  Disciplines 
Coordinate  with  Supplier 
Manage  Risk 
Manage  Data 
Manage  Configurations 
Ensure  Quality 

Systems  Engineering  Environment  Focus  Areas 

Define  and  Improve  the  Systems  Engineering  Process 
Manage  Competency 
Manage  Technology 

Manage  Systems  Engineering  Support  Environment 
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The  Integrated  Product  Development  Capability  Maturity  Model  (IPD  CMM)  grew  out  of  a  study  of  commercial  and  defense 
organizations.  The  study  focused  on  organizations  practicing  IPD  with  teams  and  conducted  interviews  for  good  and  bad  examples  of 
IPD  implementation.  The  study  team  was  searching  for  IPPD  best  practices  with  the  benefits  gained  and  the  problems  confronted  in 
its  implementation.  The  results  were  compiled  in  a  data  base  and  published  by  one  of  the  study  participants. 

The  IPD  CMM  was  used  as  a  source  model  for  the  CMMI  SE/SW/IPPD  model  (see  pages  76-79).  IDA  participated  in  the 
development  of  the  IPD  CMM. 


39 


Kerinia  Cusick,  “A  Collection  of  Integrated  Product  Development  Lessons  Learned,”  INCOSE  Conference  Proceedings,  1997. 
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Integrated  Product  Development 
Capability  Maturity  Model  (IPD-CMM) 

Product  of  SEI,  government,  and  industry 
Purpose/Scope 

Provide  a  requirements  framework  for  establishing, 
appraising,  and  improving  any  organization’s  product  life  cycle 
and  supporting  processes 

Provide  a  common  language  and  resource  for  IPD  concepts 

Support  the  adoption  of  IPD  in  a  wide  variety  of  industries, 
service  operations,  government,  and  academia 

Background/Status 

Versions  in  1996,  1997,  1998 
Merged  into  CMMI-SE/SW/IPPD 
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The  process  areas  in  the  IPD  CMM  are  grouped  into  Product  Life  Cycle,  Process  Management,  and  Integration  categories. 
Process  Management  processes  relate  to  operational  efficiency,  Integration  Processes  relate  to  effectiveness,  and  both  are  applied  to 
any  system  life  cycle. 

The  maturity  levels  of  organizational  performance  using  the  IPD  CMM  are: 

1 .  Performed  Informally 

2.  Planned  and  Tracked,  reducing  local  chaos 

3.  Well  Defined,  defined  and  tailored  processes 

4.  Quantitatively  Managed,  managing  by  facts 

5.  Continually  Improving,  optimizing  operations 
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PD-CMM  List  of  Processes 


Purpose,  Goals,  and  Best  Practices  | 

Product  Life  Cycle 

Product  Selection 
Product  Life  Cycle  Definition 
Product  Requirements  Evolution 
Solution  Design 

Product  Build,  Verification  &  Test 
Product  Support  and  Retirement 

Integration 
Project  Leadership 
Leadership  Mechanisms 
Work  Environment 
Team  Environment 
Shared  Vision 
Organizational  Leadership 
Product  Line  Evolution 
Organizational  Environment  Adaptation 


Process  Management 

Process  Planning 
Configuration  Management 
Ensuring  Quality 
Process  Monitoring  &  Control 
Organization  Training  Program 
Organization  Process  Definition 
Organization  Process  Focus 
Quantitative  Techniques 
Process  Change  Management 
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The  Capability  Maturity  Model-Integration  (CMMI)  is  actually  a  framework  that  can  be  used  to  produce  various  combinations 
of  models  based  on  the  disciplines  for  which  models  exist.  Currently,  CMMI  models  exist  for  Systems  Engineering  (SE),  Software 
(SW),  and  Integrated  Product  and  Process  Development  (IPPD).  Acquisition,  safety,  security,  and  modeling  and  simulation  are  all 
additional  disciplines  that  have  been  investigated  for  addition  to  the  model  suite. 

The  CMMI  model  integrates  the  Systems  Engineering,  Software  Engineering,  and  Integrated  Product  and  Process 
Development  (IPPD)  Capability  Maturity  Models  (CMMs)  to  provide  a  framework  for  process  integration  and  improvement  across  an 
organization.  The  CMMI  will  be  used  by  both  government  and  industry  for  self-assessments  to  increase  their  process  capability  and 
organizational  maturity.  Since  the  DoD’s  goal  is  to  select  contractors  with  a  mature  development  process  and  capability,  policy 
implications  for  a  contractor’s  use  of  the  CMMI  model  need  to  be  investigated. 

The  Software  CMM  uses  a  staged  representation.  The  Systems  Engineering  Capability  Model  (SECM),  EIA/IS  731,  uses  a 
continuous  representation.  In  a  staged  representation,  specific  process  areas  are  required  for  each  maturity  level  and  an  organization 
gets  rated  on  its  maturity.  In  a  continuous  representation,  the  organization  chooses  the  set  of  process  areas  that  applies  to  its  business 
objectives  and  then  each  process  area  gets  individually  assessed  to  a  capability  level.  The  CMMI  has  both  a  staged  representation  and 
a  continuous  representation.  An  Equivalent  Staging  Diagram  published  in  the  CMMI  models  shows  an  equivalence  between  the  two 
representations.  To  be  equivalent  to  a  Maturity  Level  3  rating  for  an  organization,  it  would  have  to  be  capability  level  3  in  all  of  the 
process  areas  listed  in  diagram. 

The  maturity  levels  within  the  CMMI  framework  are:  1)  Initial,  2)  Managed,  3)  Defined,  4)  Quantitatively  Managed,  and 
5)  Optimizing.  The  capability  levels  are  the  same  except  that  a  process  can  be  at  capability  level  0,  Incomplete,  as  well.  A  maturity 
level  for  an  organization  of  “incomplete”  does  not  make  sense  in  the  context  of  the  CMMI. 

While  the  CMMI-SE/SW/IPPD  model  was  being  developed,  a  need  was  perceived  for  a  model  version  with  the  “acquisition” 
discipline  added,  and  an  attempt  was  made  to  incorporate  the  Software  Acquisition  CMM.  This  effort  evolved  into  a  more  succinct 
addition  of  a  supplier  sourcing  (SS)  discipline,  to  create  a  CMMI-SE/SW/IPPD/SS  model. 
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Capability  Maturity  Model- 
Integration  (CMMI) 

Product  of  SEI  with  government  and  industry 
participation 

Purpose/Scope 

Combine  into  a  single  model  for  use  by  organizations  pursuing 
enterprise-wide  process  improvement 
SW-CMM  v2.0  draft  C 
EIA/IS  731 
IPD-CMM  v0.98 

Background/Status 

CMMI-SE/SW  v  1 .0  released  August  2000 
CMMI-SE/SW/IPPD  v.  1.01  released  November  2000 
CMMI-SE/SW/IPPD  v.  1.1  released  January  2002 
CMMI-SE/SW/IPPD/SS  v.  1.1  released  March  2002 
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The  CMMI  groups  processes  into  four  categories:  Process  Management,  Project  Management,  Engineering,  and  Support.  For 
each  process  area,  required  components  are  the  specific  and  generic  goals  and  expected  components  are  the  specific  and  generic 
practices.  The  required  and  expected  model  components  are  those  things  that  correspond  to  what  is  “normative”  in  standards 
tenninology.  The  other  model  components — purpose  statement,  typical  work  products,  subpractices,  amplifications,  elaborations,  and 
notes — are  all  informative  material. 

The  process  areas  and  all  required  and  expected  material  is  exactly  the  same  for  CMMI-SW,  CMMI-SE,  and  CMMI-SE/SW. 
The  only  difference  is  that  if  you  were  to  publish  CMMI-SW  or  CMMI-SE  alone,  the  informative  amplifications  for  the  other 
discipline  would  not  appear.  The  difference  is  in  informative  examples  (amplifications)  only. 

Adding  the  IPPD  discipline  to  any  model  adds  two  process  areas  (Integrated  Teaming  and  Organizational  Environment  for 
Integration)  and  expands  (adds  two  Specific  Goals)  to  the  Integrated  Project  Management  process  area. 

IDA  was  a  member  of  the  CMMI  Integrated  Product  Team  and  actively  participated  in  the  IPPD,  Engineering,  Editor,  and 
Framework  teams.  IDA  also  served  on  the  CMMI  Change  Control  Board  and  Steering  Group. 
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CMMI-SE/SW/IPPD  List  of 
Processes 

Purpose,  Goals,  and  Practices 


Project  Management  Process  Areas 

Project  Planning 
Project  Monitoring  and  Control 
Supplier  Agreement  Management 
Integrated  Project  Management 
Risk  Management 
Integrated  Teaming 
Quantitative  Project  Management 

Engineering  Process  Areas 

Requirements  Management 
Requirements  Development 
Technical  Solution 
Product  Integration 
Verification 
Validation 


Process  Management  Process  Areas 

Organizational  Process  Focus 
Organizational  Process  Definition 
Organizational  Training 
Organizational  Process  Performance 

Support  Process  Areas 

Configuration  Management 
Process  and  Product  Quality  Assurance 
Measurement  and  Analysis 
Decision  Analysis  and  Resolution 
Organizational  Environment  for 
Integration 

Causal  Analysis  and  Resolution 
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The  integrated  Capability  Maturity  Model  (iCMM)  is  a  product  of  the  Federal  Aviation  Administration  (FAA).  “The  FAA  has 
been  achieving  more  effective  and  efficient  processes  and  process  improvement  by  using  the  FAA  integrated  Capability  Maturity 
Model®  (FAA-iCMM®)  to  guide  its  improvement  efforts.”  The  authors  include  Linda  Ibrahim,  Bill  Bradford,  David  Cole,  Larry 
LaBruyere,  Heidi  Leinneweber,  Dave  Piszczek,  Natalie  Reed,  Mike  Rymond,  Dennis  Smith,  Michael  Virga,  and  Curt  Wells, 

Version  1.0  of  the  iCMM  integrates  the  CMMs  for  Software,  Systems  Engineering,  and  Software  Acquisition.  Version  2.0  of 
the  iCMM  builds  on  the  integration  concept  by  integrating  the  following  additional  standards  and  models:  ISO  9001:2000,  EIA/IS 
731,  Malcolm  Baldrige  National  Quality  Award  and  President's  Quality  Award  criteria,  CMMI-SE/SW/IPPD  and  CMMI-A,  ISO/IEC 
TR  15504,  ISO/IEC  12207,  and  ISO/IEC  CD  15288.40 

The  capability  Levels  are 

0.  Incomplete 

1 .  Perfonned 

2.  Managed,  Planned  and  Tracked 

3.  Defined 

4.  Quantitatively  Managed 

5.  Optimizing 


40 


Federal  Aviation  Administration  (FAA),  Office  of  the  Assistant  Administrator  for  Information  Services  and  Chief  Information  Officer  (AIO),  Process 
Engineering  web  site,  http://www.faa.gOv/aio/ProcessEngr/iCMM/index.htm#tr. 
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integrated  Capability  Maturity 
Model  (iCMM) 


Product  of  the  Federal  Aviation  Administration 
(FAA) 

>  Purpose/Scope 

Framework  for  achieving  an  enterprise-wide 
approach  to  process  improvement 

Background/Status 

Version  1.0  release  November  1997 

Integrated  CMMs  for  Software,  Systems  Engineering,  and 
Software  Acquisition 

FAA-iCMM  V.  2.0,  An  Integrated  Capability  Maturity 
Model  for  Enterprise-wide  Improvement,  published 
September  2001 
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The  processes  included  in  Version  2  of  the  iCMM  are  shown  on  the  accompanying  chart.  The  23  processes  can  be  grouped  into 
three  categories:  Management  Processes,  Life  Cycle  Processes,  and  Support  Processes. 
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FAA  iCMM  List  of  Processes 

..  .  Purpose,  Goals,  and  Base  Practices  I 

Mananpmpnt  r  ’  1 

Integrated  Enterprise  Management 

Project  Management 

Support 

Risk  Management 

Outsourcing 

Supplier  Agreement  Management 

Alternatives  Analysis 

Integrated  Teaming 

Measurement  and  Analysis 

Life  Cycle 

Quality  Assurance  and 

Needs 

Management 

Requirements 

Configuration  Management 

Design 

Information  Management 

Design  Implementation 

Process  Definition 

Integration 

Process  Improvement 

Deployment,  Transition,  Disposal 

Training 

Operation  and  Support 

Innovation 

Evaluation 
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The  next  section  discusses  process  improvement  models  that  aren’t  explicitly  capability  or  maturity  models. 
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Process  Improvement  Models 


Lean  Enterprise  Model 
Baldrige  National  Quality  Program 
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The  Lean  Enterprise  Model  (LEM)  was  developed  by  the  Lean  Aerospace  Initiative.  This  initiative  was  formed  at  the 
Massachusetts  institute  of  Technology  (MIT)  with  Air  Force  sponsorship  and  industry  membership. 
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Lean  Enterprise  Model  (LEM) 

Product  of  Lean  Aerospace  Initiative 
>  Purpose 

Framework  consisting  of  lean  principles,  metrics,  and 
overarching  practices 

Organizational  tool  for  MIT  and  external  information 
on  lean  principles  and  practices  from  surveys,  case 
studies,  and  other  research  activities 

Background/Status 
•  Architecture  released  July  1998 
On-line  tool  for  members  continually  updated 
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The  Overarching  Practices  for  the  LEM  are  shown  on  the  chart.  Within  each  Overarching  Practice  are  Enabling  Practices. 
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LEM  List  of  Processes 


Overarching^^Practicesj^ 

Identify  and  optimize  information  flow 

Assure  seamless  information  flow 

Optimize  capability  and  utilization  of  people 

Make  decisions  at  lowest  possible  level 

Implement  integrated  product  and  process  development 

Develop  relationships  based  on  mutual  trust  and  commitment 

Continuously  focus  on  the  customer 

Promote  lean  leadership  at  all  levels 

Maintain  challenge  of  existing  processes 

Nurture  a  learning  environment 

Ensure  process  capability  and  maturation 

Maximize  stability  in  a  changing  environment 
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The  Baldrige  National  Quality  Program  is  run  by  the  National  Institute  of  Standards  and  Technology  (NIST). 
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Baldrige  National  Quality 
Program 


Product  of  the  National  Institute  of  Standards 
and  Technology  (NIST) 

Purpose/Scope 

Improve  U.S.  organizational  performance 
Facilitate  sharing  of  best  practices 

Background/Status 
In  existence  for  14  years 
2002  criteria  available  now 
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The  Baldrige  National  Quality  Program  criteria  are  arranged  in  seven  categories  as  shown  and  detailed  below 

1 .  Leadership 

—  Organizational  Leadership 

—  Public  Responsibility  and  Citizenship 

2.  Strategic  Planning 

—  Strategy  Development 

—  Strategy  Deployment 

3.  Customer  and  Market  Focus 

—  Customer  and  Market  Knowledge 

—  Customer  Relations  and  Satisfaction 

4.  Information  and  Analysis 

—  Measurement  and  Analysis  of  Organizational  Perfonnance 

—  Information  Management 

5.  Human  Resource  Focus 

—  Work  Systems 

—  Employee  Education,  Training,  and  Development 

—  Employee  Well-Being 

6.  Process  Management 

—  Product  and  Service  Processes 

—  Business  Processes 

—  Support  Processes 

7.  Business  Results 

—  Customer-Focused  Results 

—  Financial  and  Market  Results 

—  Organizational  Effectiveness  Results 
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Baldrige  National  Quality 
Program  Criteria 


Leadership 
Strategic  Planning 
Customer  and  Market  Focus 
Information  and  Analysis 
Human  Resource  Focus 
Process  Management 
Business  Results 
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The  next  section  describes  the  various  appraisal,  evaluation,  and  assessment  methods  that  accompany  the  models  covered  in 
the  preceding  sections. 
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Appraisal  Methods 

Software  Process  Assessment  (SPA) 

Process  Assessment  (ISO/I EC  15504) 

Capability  Maturity-based  Appraisal  for  Internal  Process  Improvement 
(CBA  IPI) 

Software  Capability  Evaluation  (SCE) 

Software  Development  Capability  Evaluation  (SDCE) 

SE-CMM  Assessment  Method  (SAM) 

>  SECAM 

SECM  Appraisal  Method  (EIA/IS  731.2) 

Standard  CMMI  Appraisal  Method  for  Process  Improvement 
(SCAMPI) 

FAA-iCMM  Appraisal  Method  (FAM) 

Lean  Enterprise  Self  Assessment  Model  (LESAT) 

Baldrige  National  Quality  Award 


95 


The  Software  Process  Assessment  (SPA)  is  the  original  assessment  method  developed  by  SEI.  It  was  developed  in  1987  based 
on  “Characterizing  the  Software  Process:  A  Maturity  Framework”  by  Watts  Humphrey.41  SEI  later  commercialized  the  model  in  1990 
so  that  it  could  be  more  widely  disseminated.42 

The  SPA  predates  the  Software  CMM.  When  the  CMM  was  released,  organizations  modified  SPA  to  reflect  it,  but  in  1994  the 
CBA  IPI  method  (see  pages  110-111)  was  developed  and  replaced  SPA  43 


41  Will  Hayes  and  Dave  Zubrow,  “Moving  on  and  up:  Data  and  Experience  Doing  CMM-Based  Process  Improvement,”  (1995), 
http://www.sei.cmii.edu/piib/documents/95.reports/pdf/tr008.95.pdf. 

42  SCE  Version  3.0  Method  Description,  http://www.sei.cmu.edu/publications/documents/96.reports/96.tr.002.html. 

43  CBA  IPI  Method  Description,  http://www.sei.cmu.edu/publications/documents/96.reports/96.tr.007.html. 
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Software  Process  Assessment 
(SPA) 


Product  of  SEI 

•  Original  SEI  process  assessment  model 

Purpose/Scope 

Assessment  method  based  on  Humphrey’s 
“Characterizing  the  Software  Process:  A  Maturity 
Framework” 

•  Used  for  internal  process  assessment 

>  Background/Status 

Developed  in  1987  and  commercialized  in  1990 
Predates  the  CMM 

SPA  has  been  replaced  by  CBA  IPI 
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The  ISO/IEC  Technical  Report  (TR)  15504,  Software  Process  Assessment,  is  the  result  of  the  Software  Process  Improvement 
Capability  dEtennination  (SPICE)  project.  The  TR  consists  of  9  parts,  is  based  on  a  process  reference  model,  and  follows  a  continuous 
representation  with  capability  levels.  Field  trials  of  the  TR  began  in  1995  and  their  results  are  contributing  to  the  revision  of  ISO/IEC 
15504.  Field  trials  will  continue  until  the  full  standard  is  published  in  the  20032004  time  frame. 

To  align  with  the  recent  addition  of  “systems”  to  the  JTC1  SC7  purview,  the  revision  of  ISO/IEC  15504  will  be  titled,  Process 
Assessment  and  will  apply  to  both  software  and  systems  assessments.  The  revision  is  proceeding  as  follows: 

•  15504-1:  Concepts  and  Vocabulary  Due  February  200415504-2,  Performing  an 

Assessment  (Requirements)  Due  July  2003 

•  1 5504-3  Guidance  on  Performing  an  Assessment  Due  September  2003 

•  15504-4:  Guidance  on  Using  the  Results  of  an  Assessment  Due  December  200315504-5:  An  Exemplar  Process 

Assessment  Model  Due  December  2004 

The  capability  dimension  of  ISO/IEC  15504  contains  the  following  levels: 

0.  Incomplete 

1 .  Performed 

2.  Managed 

3.  Established 

4.  Predictable 

5.  Optimizing 
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Process  Assessment 
(ISO/IEC  15504) 


> Product  of  ISO/IEC  JTC1  SC7  WG10 

Originally  sponsored  by  the  SPICE  project 

>  Purpose/Scope 

Provides  a  framework  for  the  assessment  of  software 
processes  and  its  use  in  the  two  contexts  of  process 
improvement  and  process  capability  determination 

>  Backg  rou  nd/Status 

Technical  Report,  1st  edition,  published  August  1998 
Software  Process  Assessment 
Consists  of  nine  parts 

Currently  in  revision  to  become  5-part  full  standard  for 
both  software  and  systems 
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The  Systems  Engineering  Capability  Maturity  Model  (SE-CMM)  Appraisal  Method,  or  SAM,  “provides  a  description  of  the 
appraisal  method  developed  for  use  with  the  SE-CMM  when  evaluating  adherence  to  the  principles  and/or  practices  of  the  SE-CMM. 
It  also  contains  the  appraisal  method  requirements.”44 

This  appraisal  method  accompanies  the  SE-CMM  model  (see  pages  64-67). 


SE-CMM  Appraisal  Method  Description,  SECMM-94-06  CMU/SEI-94-HB-05. 
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SE-CMM  Appraisal  Method  (SAM) 

Product  of  EPIC 
Purpose/Scope 

Provides  a  description  of  the  appraisal  method  for  the 
SE-CMM  when  evaluating  adherence  to  its  principles 
and  practices.  It  also  contains  the  appraisal  method 
requirements. 

Background/Status 

Version  1.0  of  SE-CMM  Appraisal  Method  (SAM) 
released  June  1995 

SE-CMM  Appraisal  Method  v.1.1  released  1996 
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At  the  same  time  that  EPIC  was  developing  the  SE-CMM,  INCOSE  was  developing  the  SECAM.  The  SECAM  consisted  of  a 
questionnaire  to  be  used  with  both  ANSI/EIA  632,  Processes  for  Engineering  a  System,  and  IEEE  1220,  Application  and  Management 
of  the  Systems  Engineering  Process. 

There  is  a  model  associated  with  the  SECAM  that  is  organized  around  19  Key  Process  Areas  (KPAs).  These  KPAs  are  in  the 
Technical,  Management,  and  Environment  areas  of  Systems  Engineering.  Although  modeled  after  the  SW-CMM,  this  model  added 
some  non-process  areas  of  concentration. 
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Systems  Engineering  Capability 
Assessment  Model  (SECAM) 

Product  of  INCOSE  Capability  Assessment  Working 
Group  (CAWG) 

Purpose/Scope 

Address  assessment  of  systems  engineering  capability 

Used  systems  engineering  standards  (IEEE  1220  and  EIA/IS  632) 
as  a  basis  for  a  questionnaire 

Used  and  modified  the  approach  of  the  CMM  for  Software 

Modification  consisted  primarily  of  adding  questions  about  product 
effectiveness  and  team  experience  as  well  as  a  few  other  non¬ 
process  topics 

Background/Status 

•  Version  1.5  released  July  1996 

Merged  with  the  SE-CMM  to  create  the  SECM  and  no  longer 
supported 
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Volume  2  of  EIA/IS  731  contains  the  Appraisal  Method,  developed  by  the  El  A  G47  SECM  Working  Group  that  developed 
EIA/IS  731.1,  the  SECM  model.  The  method,  like  the  model,  was  the  result  of  an  initiative  to  merge  the  SE-CMM  (pages  6466  and 
100-101)  and  the  SECAM  (pages  102-103).  It  is  helpful  for  industry-wide  baselines  and  comparisons  for  benchmarking  systems 
engineering  processes. 

The  EIA/IS  731  appraisal  method  is  based  on  a  continuous  architecture,  as  opposed  to  the  appraisal  methods  associated  with 
the  Software  CMM  (CBA  IPI  and  SCE,  see  pages  1 10-1 13),  which  are  based  on  a  staged  architecture.  The  intent  of  the  EIA/IS  731.2 
SECM  Appraisal  Method  is  to  support  self-improvement  rather  than  an  official  evaluation  or  audit.  The  continuous  architecture  is 
especially  conducive  to  self-improvement  within  the  organization’s  business  strategy.  The  method  involves  comparing  the 
organization’s  processes  with  the  focus  areas,  and  specific  and  generic  practices  of  the  EIA/IS  731.1  SECM. 
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SECM  Appraisal  Method 
(EIA/IS  731.2) 


> Product  of  EIA,  INCOSE,  and  EPIC 
>  Purpose/Scope 

“To  support  the  development  and  improvement  of 
systems  engineering  capability” 

>Background/Status 

Reflects  March  1996  initiative  to  merge  SECAM  and 
its  Assessment  Method  with  that  of  SE-CMM  and  its 
Appraisal  Method 

Intended  to  complement  the  use  of  EIA  632  and 
IEEE  1220 

EIA731,  Draft  Version  1.0 
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The  CMM  Appraisal  Framework  (CAF)  was  developed  by  the  SEI  to  identify  the  requirements  and  desired  characteristics  of  a 
CMM-based  appraisal  method.  The  framework  was  designed  to  ensure  that  the  CMM-based  appraisals  were  consistent  and  reliable. 
CAF  version  1.0  was  published  in  February  1995  as  Capability  Maturity  Model-based  Appraisal  for  Internal  Process  Improvement 
(CBA  IPI)  Method  Description  4.  The  Software  CMM  version  1.1  is  the  associated  reference  model.45 

Both  the  CBA  IPI  and  SCE  (pages  1 10-1 13)  version  3.0  were  designed  to  be  CAF  compliant,  which  means  that  their  results 
should  be  consistent. 


45  CAF  Version  1.0,  3,  http://www.sei.cmu.edu/publications/documents/95.reports/95-tr-001/95-tr-001-abstract.html. 


106 


CMM  Appraisal  Framework 
(CAF) 

Product  of  SEI 
Purpose/Scope 

Identifies  the  requirements  and  desired 
characteristics  of  a  CMM-based  appraisal 
method 

CMM  Version  1.1  is  associated  reference  model 

Similar  to  ARC  for  CMMI 

CBA  IPI  and  SCE  both  CAF  compliant 

Background/Status 

CAF  Version  1.0  published  February  1995 
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The  IDEAL  model  is  an  improvement  model  that  was  developed  by  SEI  to  aid  organizations  in  improving  their  software 
processes.  It  is  named  for  the  five  phases  that  an  organization  following  this  model  would  go  through:  Initiating,  Diagnosing, 
Establishing,  Acting,  and  Learning.  Many  of  the  services  that  SEI  provides  follow  the  IDEAL  model.46 


46  SEI  Web  site,  The  Ideal  Model,  http://www.sei.cmu.edu/ideal/ideal.html. 
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IDEAL  Model 


Life  cycle  model  for  continuous  software  process  improvement 

Consists  of  5  Phases:  Initiating,  Diagnosing,  Acting, 
Establishing,  and  Leveraging 

IDEAL  strategy  is  used  in  many  of  SEI’s  services 

Learning 


Establishing 
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CBA  IPI  (Capability  Maturity  Model-based  Appraisal  for  Internal  Process  Improvement)  is  a  CMM-based  assessment  method 
that  was  developed  by  SEI.47  It  is  a  method  designed  to  determine  and  improve  the  state  of  one’s  own  processes,  as  opposed  to  the 

4o 

Software  Capability  Evaluation  (SCE,  pages  1 12-1 13)  that  is  intended  for  use  when  evaluating  another  organization’s  processes. 

CBA  IPI  is  CAF  (CMM  Appraisal  Framework)  compliant,  which  means  that  it  adheres  to  the  standards  required  for  a  CMM- 
based  appraisal.  CBA  IPI  also  uses  the  IDEAL  approach  for  process  improvement. 

The  first  version  of  CBA-IPI  (Version  1.0)  was  published  in  May  1995  in  response  to  user  needs  for  a  CMM-based  appraisal 
method.  Before  CMM  and  the  CBA-IPI  method,  organizations  used  the  SPA  (Software  Process  Assessment)  method  (pages  96-97)  to 
assess  their  software  processes.  After  the  CMM  was  published  in  1991,  organizations  modified  the  SPA  method  to  reflect  CMM,  but 
CBA-IPI  was  the  first  SEI-developed  assessment  based  on  the  CMM.  The  CBA-IPI  method  was  updated  to  Version  1.1  in  March 
1996.  49 


47  CBA  IPI  Method  Description,  http://www.sei.cmu.edu/publications/documents/96.reports/96.tr.007.html 

48  SEI  website,  “SEI  Appraiser  Program,”  www.sei.cmu.edu/managing/app.directory.html. 
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Capability  Maturity  Model  -  based  Appraisal  for 
Internal  Process  Improvement  (CBA  IPI) 

Product  of  SEI 
>  Purpose/Scope 

Developed  in  response  to  user  needs  for  a  CMM- 
based  assessment  method 

CMM-SW  Version  1.1  is  its  reference  model 

Used  for  assessments  of  one’s  own  processes  (as 
opposed  to  SCE) 

Uses  IDEAL  approach 

CAF  (CMM  Appraisal  Framework)  compliant 

Background/Status 

Version  1.0  released  in  1995 
Version  1.1  released  in  1996 


in 


The  Software  Capability  Evaluation  (SCE),  developed  by  SEI,  is  an  evaluation  method  used  for  software  acquisition.  The  most 
recent  version,  3.0,  reflects  CMM  Version  1.1. 50  It  is  currently  one  of  the  two  assessment  methods  that  are  approved  evaluation  tools 
for  contractor  selection  under  ACAT  1  Acquisition  Policy.51 

SCEs  are  used  to  detennine  the  state  of  another  organization’s  process,  rather  than  one’s  own  processes,  although  they  may  be 

52 

used  internally  to  prepare  for  an  external  evaluation. 

The  most  recent  version  (Version  3.0)  of  the  SCE  method  was  published  in  1996.  The  original  version  was  described  in 
A  Method  for  Assessing  the  Software  Engineering  Capability  of  Contractors  (1987),  which  was  developed  to  support  supplier 
selection  for  government  software  acquisition.  The  evolution  of  CMM  and  CAF  led  to  SCE  Version  1.5  (July  1993),  and  the  method 
was  later  updated  to  comply  with  CMM  Version  1 . 1  in  SCE  Version  2.0  (June  1994). 53 


50  CBA  IPI  Method  Description,  p.  4. 

51  Jack  Ferguson,.  “DoD  Acquisition  Policy  and  SEI  CMM  Level  3”  (presentation),  DTIC  Web  site,  http://www.dtic.mil/ndia/systems/Ferguson2.pdf. 

52  SEI  Web  site,  “SEI  Appraiser  Program,”  www.sei.cmu.edu/managing/app.directory.html. 
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Software  Capability  Evaluation 
(SCE) 

Product  of  SEI 
Purpose/Scope 

Developed  to  support  source  selection  in  major 
government  software  acquisition  and  also  used  for 
evaluation  of  internal  processes 

Used  to  determine  the  state  of  another 
organization’s  processes  (as  opposed  to  CBA  IPI) 

Version  3.0  is  CAF  compliant 

Background/Status 

First  version  described  in  A  Method  for  Assessing 
the  Software  Engineering  Capability  of  Contractors 

SCE  Version  3.0  is  latest  version,  published  in  April 
1996 
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SDCE  was  developed  by  ASC/EN  based  on  two  other  assessment  methods,  SCE  and  ASC/EN  Software  Development 
Capability/Capacity  Review.  The  evaluation  method  was  initiated  in  1992. 54 

As  described  in  Pamphlet  63-103, 55  the  evaluation  comprises  two  parts:  the  model  and  the  application  process.  The  model 
contains  six  functional  areas:  Program  Management,  Systems  Engineering,  Software  Engineering,  Quality  Management  and  Product 
Control,  Organizational  Resources  and  Program  Support,  and  Program  Specific  Technologies.  These  are  further  broken  into  Critical 
Capability  Areas  (CCAs),  and  then  Critical  Capabilities,  which  are  made  up  of  open-ended  questions  and  criteria.56 

Application  of  SDCE  is  fully  integrated  into  the  source  selection  process.  SDCE  is  used  to  evaluate  the  contractors’  abilities  to 
develop  the  software  defined  in  the  specific  Request  for  Proposal  (RFP).57 

SDCE  is  currently  one  of  the  two  assessment  methods  that  are  approved  evaluation  tools  for  contractor  selection  under 
ACAT  1  Acquisition  Policy.58 


54  Philip  Babel.  “Software  Development  Capability  Evaluation:  An  Integrated  Systems  and  Software  Approach,”  section  heading:  Objectives, 
http://www.stsc.hill.af.mil/crosstalk/1997/apr/development.asp. 

55  AFMC  Pamphlet  63-103,  .Volumes  1  and  2,  Software  Development  Capability  Evaluation,  15  June  1994. 

56  Ibid.,  section  heading:  Model. 

57  “SDCE”  Software  Consortium  Web  site,  http://www.software.org/quagmire/descriptions/sdce/asp. 

58  Jack  Ferguson,  Briefing,  DoD  Acquisition  Policy  and  SEI CMM Level  3,  DTIC  Web  site,  http://www.dtic.mil/ndia/systems/Ferguson2.pdf. 
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Software  Development 
Capability  Evaluation  (SDCE) 

Product  of  Aeronautical  Systems  Center 
Engineering  Directorate  (ASC/EN) 

Purpose/Scope 

Purpose  is  to  reduce  acquisition  risk  for 
software-intensive  systems 

SDCE  is  conducted  as  a  part  of  the  source  selection 
process 

Background/Status 

•  Effort  began  1 992 

Result  of  merger  between  SCE  and  the  ASC/EN  Software 
Development  Capability/Capacity  Review 

Pamphlet  63-103  describing  SDCE  published  in  1994 
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The  Appraisal  Requirements  for  CMMI  (ARC)  provides  a  set  of  criteria  for  developing,  defining,  and  using  assessment 
methods  based  on  the  CMMI  model.59  ARC  appears  to  be  analogous  to  the  CAF,  which  identifies  requirements  and  characteristics  of 
CMM-based  assessments.  ARC  defines  three  different  classes  of  assessments,  A,  B,  and  C,  where  A  is  the  most  rigorous  assessment 
and  C  is  a  “quick  look.”60 


59  CMMI  Version  1.1,  http://www.sei.cmu.edu/cmmi/products/models.html. 

60  ARC  Version  1.0,  chapter  3,  http://www.sei.cmu.edu/publications/documents/00.reports/00tr01  l.html. 
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Appraisal  Requirements  for 
CMMI  (ARC) 


Product  of  SEI 

Authored  by  CMMI  Product  Team 

>  Purpose/Scope 

Defines  requirements  considered  essential  to 
appraisal  methods  intended  for  use  with  CMMI 
models 

Defines  appraisal  classes  based  on  appraisal 
usage  scenarios 

Background/Status 

Version  1.0  released  August  2000 
Version  1.1  released  December  2001 
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The  Standard  CMMI  Appraisal  Method  for  Process  Improvement  (SCAMPI)  is  a  product  of  the  SEI,  developed  by  the 
Assessment  Method  Integrated  Team  (AMIT).  SCAMPI  is  a  method  that  meets  all  the  Class  A  assessment  requirements  for  CMMI 

defined  by  ARC  version  1.0. 61  SCAMPI  uses  the  IDEAL  approach  for  process  improvement.  This  approach  consists  of  5  phases: 

62 

Initiating,  Diagnosing,  Establishing,  Acting  and  Learning. 

SCAMPI  is  based  on  two  earlier  assessment  methods,  CBA  IPI  version  1.1  and  EIA/IS  73 1.2. 63 

The  first  version  of  SCAMPI  (version  1.0)  was  published  in  October  2000.  The  Assessment  Method  Integrated  Team  (AMIT) 
evolved  the  model  to  version  1.1,  which  was  published  by  the  SEI  in  December  2001.  The  SEI  will  no  longer  publish  updates  to  the 
CBA  IPI  or  the  SCE.  “CBA  IPI  Lead  Assessors  and  SCE  Lead  Evaluators  will  be  trained  through  December  2003;  however, 
authorized  Lead  Assessors  and  Lead  Evaluators  will  need  to  transition  to  SCAMPI  Lead  AssessorsSM  within  2  years  of  the  termination 
of  CBA  IPI  and  SCE  Lead  Assessor/Evaluator  training.  SCAMPI  will  then  be  the  appraisal  method  of  choice.”64 


61  SCAMPI  Method  Description,  p.  3. 

62  Ibid.,  2 

63  SCAMPI  Method  Description,  p.  xi. 

64  How  Will  Sunsetting  of  the  Software  CMM  be  Conducted?,  SEI  Web  site,  http://www.sei.cmu.edu/cmmi/comm/sunset.html. 
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Standard  CMMI  Appraisal  Method 
for  Process  Improvement  (SCAMPI) 

> Product  of  SEI 

Authored  by  Assessment  Method  Integrated  Team 
(AMIT) 

>  Purpose/Scope 

Provides  approach  to  conduct  an  in-depth  class  A 
assessment  satisfying  all  assessment  requirements 
for  CMMI  (ARC  version  1.0) 

Based  on  CBA  IPI  Version  1.1  and  EIA/IS  731.2 
Uses  IDEAL  approach 

Background/Status 

SCAMPI  Version  1.0  published  in  2000 

Scampi  Method  Definition  Document  V.  1.1  published 
December  2001 
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The  FAA-iCMM  Appraisal  Method  (FAM)  provides  the  “appraisal  method  for  comparing  processes  being  practiced  by  an 
organization  to  those  in  the  iCMM.”  65  The  authors  include  Linda  Ibrahim,  Larry  LaBruyere,  Pete  Malpass,  John  Marciniak,  Art 
Solomon,  and  Chuck  Weigh  The  FAM  also  includes  five  variations  of  the  method  to  meet  various  appraisal  needs.  The  total  of  6 
appraisal  methods  include:66 

•  Standard,  full  internal  FAM  framework  appraisal,  similar  to  CBA-IPI 

•  Facilitated  discussion 

•  Document-intensive 

•  Questionnaire-based 

•  Interview-based 

•  Full  external  evaluation 


65  http://www.faa.gov/aio/ProcessEngr/iCMM/index.htm 

66  Linda  Ibrahim,  “Smart  Buying  with  the  Federal  Aviation  Administration’s  Integrated  Capability  Maturity  Model,”  Crosstalk,  Vol.  11,  No.  11,  November 
1998,  pp.  1520. 

Linda  Ibrahim,  “Using  an  Integrated  Capability  Maturity  Model — The  FAA  Experience,”  Proceedings  of  the  Tenth  Annual  International  Symposium  of  the 
International  Council  on  Systems  Engineering  (INCOSE),  Minneapolis,  Minnesota,  July  2000,  pp.  643648. 
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FAA-iCMM  Appraisal  Method 
(FAM) 


Product  of  the  Federal  Aviation  Administration 
(FAA) 

Scope 

Full  appraisal  method  for  processes  being 
practiced  to  the  iCMM 

Five  variations  to  satisfy  various  appraisal  needs 

Background/Status 

Version  1.0  released  April  1999 
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The  Lean  Enterprise  Self-Assessment  Tool  (LESAT)  was  developed  by  the  Lean  Aerospace  Initiative  for  the  purposes  of 
enterprise  self-assessment.  It  is  produced  in  two  volumes:  the  Guide  and  the  Maturity  Matrices. 
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Lean  Enterprise  Self-  Assessment 
Tool  (LESAT) 


Product  of  Lean  Aerospace  Initiative 

Purpose/Scope 

Enterprise-level  self-assessment 

Two  volumes 
LESAT  Guide 
LESAT  Maturity  Matrices 

Background/Status 

Developed  and  field-tested  over  18  months 
Version  1.0  released  August  2001 
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The  Baldrige  National  Quality  Award  is  based  on  an  assessment  of  the  Baldrige  Criteria  shown  on  pages  9293.  The  maximum 


points  that 

can  be  awarded  are  shown  below,  for  a  maximum  possible  total  score  of  1000  points. 

1. 

Leadership 

120  points 

2. 

Strategic  Planning 

85  points 

3. 

Customer  and  Market  Focus 

85  points 

4. 

Information  and  Analysis 

90  points 

5. 

Human  Resource  Focus 

85  points 

6. 

Process  Management 

85  points 

7. 

Business  Results 

450  points 
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Baldrige  National  Quality  Award 

Product  of  the  National  Institute  of  Standards 
and  Technology  (NIST) 

Administered  with  the  American  Society  for  Quality 
(ASQ) 

>  Purpose/Scope 

Foster  success  of  the  Baldrige  National  Quality 
Program 

Improve  U.S.  competitiveness 

Winners  share  information  in  annual  conference 

Background/Status 

In  existence  for  14  years 
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LIST  OF  ACRONYMS  AND  ABBREVIATIONS 


ACAT 

AIA 

AMIT 

ANSI 

ARC 

ASC/EN 

CAF 

CAWG 

CBA  IPI 

CD 

CMM 

CMM-SW 

CMMI 

DoD 

DUSD 

EIA 

EPIC 


Acquisition  Category 

Aircraft  Industry  Association 

Assessment  Method  Integrated  Team 

American  National  Standards  Institute 

Appraisal  Requirements  for  CMMI 

Aeronautical  Systems  Center  Engineering  Directorate 

CMM  Appraisal  Framework 

Capability  Assessment  Working  Group 

Capability  Maturity  Model-based  Appraisal  for  Internal  Process  Improvement 

Committee  Draft 

Capability  Maturity  Model 

Capability  Maturity  Model  for  Software 

Capability  Maturity  Model-Integration 

Department  of  Defense 

Deputy  Under  Secretary  of  Defense 

Electronic  Industries  Alliance 

Enterprise  Process  Improvement  Collaboration 


A-l 


ESPRIT 

FAA 

FCD 

FDAM 

FDIS 

IDEAFSM 

IEC 

IEEE 

INCOSE 

IPD 

IPD-CMM 

IS 

ISO 

JTC 

LAI 

LEM 

LESAT 

MIL- STD 


European  Software  Program  for  Research  in  Information  Technologies 

Federal  Aviation  Authority 

Final  Committee  Draft 

Final  Draft  Amendment 

Final  Draft  International  Standard 

Initiating,  Diagnosing,  Establishing,  Acting  and  Learning  model 
International  Electrotechnical  Commission 
Institute  of  Electrical  and  Electronic  Engineers 
International  Council  on  Systems  Engineering 
Integrated  Product  Development 

Integrated  Product  Development  Capability  Maturity  Model 
Interim  Standard 

International  Organization  for  Standardization 

Joint  Technical  Committee 

Lean  Aerospace  Initiative 

Lean  Enterprise  Model 

Lean  Enterprise  Self-Assessment  Tool 

Military  Standard 


A-2 


MIT 

Massachusetts  Institute  of  Technology 

NIST 

National  Institute  of  Standards  and  Technology 

NSIA 

National  Security  Industries  Association 

PDTR 

Proposed  Draft  Technical  Report 

SC 

Subcommittee 

SCAMPI 

Standard  CMMI  Appraisal  Method  for  Process  Improvement 

SCE 

Software  Capability  Evaluation 

SE 

Systems  Engineering 

SEI 

Software  Engineering  Institute 

SEP 

Systems  Engineering  Process 

SPA 

Software  Process  Assessment 

SPICE 

Software  Process  Improvement  Capability  dEtermination 

sw 

Software 

TAG 

Technical  Advisory  Group 

TC 

Technical  Committee 

TG 

Technical  Group 

TR 

Technical  Report 

WD 

Working  Draft 

A-3 


WG 

Model  Names 

ANSI/EIA  632 
EIA/IS  731 
iCMM 
IEEE  1220 
IEEE/EIA  12207 
ISO  9000:2000 
ISO  9000:2000 
ISO  9001:2000 
ISO  9004:2000 
ISO  19011 
ISO/IEC  12207 
ISO/IEC  15288 
ISO/IEC  TR  15504 
ISO/IEC  15504 
SAM 
SDCE 


Working  Group 

Processes  for  Engineering  a  System 

Systems  Engineering  Capability  Model  (SECM),  Interim  Standard 
Integrated  Capability  Maturity  Model 

Application  and  Management  of  the  Systems  Engineering  Process 
Industry  Implementation  of  International  Standard  ISO/IEC  12207:1995 
Family  of  Standards  encompassing  ISO  9000,  ISO  9001,  ISO  9004  and  ISO  19011 
Quality  Management  Systems-Fundamentals  and  Vocabulary 
Quality  Management  Systems-Requirements 

Quality  Management  Systems-Guidelines  for  Perfonnance  Improvement 

Guidelines  for  Auditing  Quality  and  Environmental  Management  Systems 

Information  Technology-Software  Life  Cycle  Processes 

Systems  Engineering-System  Life  Cycle  Processes 

Information  Technology-Software  Process  Assessment,  Technical  Report 

Information  Technology-Process  Assessment 

SE-CMM  Assessment  Method 

Software  Development  Capability  Evaluation 


A-4 


SE-CMM 
SECAM 
SECM 
SW-CMM 
US  12207 


Systems  Engineering-Capability  Maturity  Model 
System  Engineering  Capability  Assessment  Model 
Systems  Engineering  Capability  Model  (EIA/IS  73 1 ) 

Capability  Maturity  Model  for  Software 

Industry  Implementation  of  International  Standard  ISO/IEC  12207:1995  (same  as  IEEE/EIA  12207) 
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